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ABSTRACT 

The emphasis on using performance assessments to 
support systematic state-, district-, or school-wide purposes is a 
recent development in the assessment-reform movement. This document 
presents data from a 3-year case study that was designed to elucidate 
the status of assessment reform in United States education systems 
and offer recommendations for policy and future research. Researchers 
made visits to 16 schools during spring 1994; they revisited 7 of the 
sites during spring 1995. Data were derived from documents, telephone 
interviews, observations, and onsite interviews. Each case-study 
summary contains four sections: a school profile and introduction, a 
description of performance assessment, the context of implementation, 
and the use and impact of performance assessment in the school. 
Appendices contain a sample mathematics task; a list of essential 
skills for reading, writing, and mathematics in grades 9-12; and 
mathematics rubrics. (LMI) 
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PREFACE 



Studies of Education Reform: Assessment of Student Performance is the result of a 
research project conducted by Pelavin Research Institute (PRI)> an affiliate of the American 
Institutes for Research (AIR), under a contract with the Office of Educational Research and 
Improvement (OERI) of the U. S. Department of Education (Contract Number RR91 172004). In 
1991, OERI issued a request for proposals entitled "Studies of Education Reform." Twelve 
studies were incorporated under this general heading, each reflecting some aspect of the reform 
movement that had placed education at the forefront of the national agenda in the late 1980s. PRI 
was awarded a three-year contract to study assessment reform, which we interpreted to mean the 
contribution of performance-based, non-multiple choice assessments to education reform. 

Contractors for all 12 reform studies were required to hold a national conference within 
the initial year of their study and to commission papers on important aspects of the reform topic. 
PRI, in collaboration with the OERI study of curriculum reform (conducted by Ronald Anderson 
of the University of Colorado), held a national conference on performance assessment and 
curriculum reform as a pre-session to the Annual Student Assessment Conference, organized by 
the Education Commission of the States, in Boulder, Colorado, in June of 1992. The assessment 
component of the pre-session conference included discussions of the content of nine 
commissioned papers that are to appear in the book, Implementing Performance Assessments: 
Promises, Problems, and Challenges (Kane & Mitchell, in press). 1 The papers, the conference 
attendees’ insights, and OERI's research questions helped us refine our study’s intellectual and 
methodological framework. 

The larger and more significant context for this study was the increasing commitment 
across the nation to performance assessment as a reform strategy. For example, California 
spearheaded the reform movement with statewide open-ended mathematics assessments in the late 
1980s, and Vermont followed suit ith its first, statewide portfolio assessments. In the late 1980s 
and early 1990s, other states, districts, and schools also began developing and implementing 
performance-based assessments. 

In the subsequent two years, we visited 16 schools across the country (of which we 
revisited seven) that were participating in the development or implementation of performance 
assessments as a result of national, state, district, or local assessment reform initiatives. In 
addition, we presented papers based upon our initial study findings at the American Educational 
Research Association conference in April, 1995, and the Council of Chief State School Officers 
conference on Large-Scale Assessment in June, 1994 and 1995. 

This volume, the second in a three-volume series, presents case studies of the 16 schools 
we visited across the country. These schools were involved in developing and implementing 
performance assessments as a result of national, state, district, or local assessment reform 
initiatives. The results of the project are contained in Volume I: Findings and Conclusions, and 
the specific research objectives and study design are presented in Volume III: Technical 
Appendix — Research Deiign and Methodology. 



1 All royalties resulting from the sales of this book will be contributed to the Leigh 
Burstein Memorial Fund, administered by the University of California at Los Angeles Foundation. 
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INTRODUCTION: 

AN OVERVIEW OF STUDY OBJECTIVES AND DESIGN 



Objective 

The major objectives of the three-year longitudinal study Studies of Education Reform: 
Assessment of Student Performance are as follows: 

Objective / : Document and analyze key characteristics of 

performance assessments; 

Objective 2 : Document and analyze facilitators and barriers in 

assessment reform; and 

O bjective 3 : Document and assess impacts of performance 

assessments on teaching and learning. 

Our ultimate purpose in this study was to elucidate the status of assessment reform in U.S. 
education systems and to offer recommendations for policy and future research. 



Research Design 1 : Case Studies 

Our research design employed a qualitative, case-study approach to collecting data about 
performance assessments and their impacts at the school level. 

We designed a modified time-series approach for gathering data, which enabled us to obtain 
both cross-sectional and longitudinal data. Cross-sectional data allowed us to make comparative 
remarks about assessments and school contexts. The longitudinal data allowed us to document the 
effects of and changes in performance assessments over time within sites. 

We selected 16 sites (the definition of “site” for this study encompasses both a performance 
assessment and a single school at which it is being used) which a team of two researchers visited a 
single time during a two-day site visit. We then selected a subset of 7 sites, which the team 
returned to for a second visit (therefore, longitudinal data were collected for only 7 of the 16 sites). 

We conducted the first set of site-visits in the Spring of 1994, and the second set of site- 
visits in the Spring of 1995. (Two of the single time site-visits were conducted in the Spring of 
1995). Exhibit 1 shows our site-visit design. 



1 A full discussion of stud objectives and design is continued in Volume III: Technical 
Appendix — Research Design and Methodology. 
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Sample Selection Criteria 



As described above, our research design called for two waves of data collection: first round 
of visits to all sites included in the sample, followed by a second round of visits to a subset of those 
sites. Bel'w, we describe the criteria we applied to select sites for inclusion in rounds one and two. 



EXHIBIT 1 
Site Visit Design 




Selection Criteria: Round One Sites 

The overarching objective of our site selection process was to identify, insofar as possible, a 
set of school sites that exhibited the range of experiences American schools are having with the 
development and implementation of performance assessments. For the purposes of our study, we 
defined a case study site as a single school where a performance assessment was being 
implemented. To select the sites, we delineated two sets of criteria - those pertaining to 
performance assessments and those pertaining to schools. 

Performance assessments are marked by a number of variable characteristics, and we 
attempted to obtain variation in our sample within each characteristic. Selection criteria pertaining 
to performance assessment characteristics included: 

• Type of Assessment Performance assessments come in a variety of forms, 
including portfolios, on-demand assessments, demonstrations and presentations, and 
extended projects. We wanted our sample to include assessments that reflected this 
range so that we might discern variation in effects of assessment type on teaching 
and learning at the local level. 

• Locus of Development . The movement toward the use of performance-based 
assessments is taking place at all levels of educatic: nal authority. States, districts, 
and schools alike are developing and implementing performance assessments. 
Furthermore, some national-level efforts, such as the New Standards Project and the 
Coalition of Essential Schools, are also influencing the turn toward performance 
assessments. Because the purposes, design, and impact of assessments developed at 
different levels of authority could potentially vary significantly, we wanted our 
sample to reflect this diversity in locus of development. 
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• Statin r»f Implementation . The entire performance assessment movement is still 
relatively young. However, performance assessments do vary with respect to their 
stage in development and implementation. Therefore, we wanted our sample to 
include assessments with varying status of implementation: developmental and pilot, 
full-scale implementation, and maintenance. 

• Content Area. Performance assessments can be used in all subject areas, but the 
assessments can look different for different subject areas. We wanted our sample to 
include assessments that focused on a range of subject areas including language arts, 
mathematics, science, and social science. 

We attempted also to obtain variation across two school characteristics: 

• School Level . We wanted the sample to include elementary schools, middle 
schools, and high schools, since performance assessments might affect teachers and 
students differently at the various levels of schooling. 

• Geographical Diversity . Because American children are educated in schools in 
50 states and the District of Columbia, and because these schools are located in 
urban, suburban, and rural areas, we wanted the sample to include school sites 
located in various regions of the country and in communities of varying urbanicity. 

Selection Criteria: Round Two Sites 

We chose a subset of seven sites for a second round of data collection. We based selection 
of the seven round two sites upon one or more of the following criteria: 

• It was anticipated that changes in the performance assessment design or 
implementation would take place between 1993-94 and 1994-95. 

• Our understanding of the effects of assessment reform at the site was less than clear 
based on one round of data collection and was likely to improve with a second visit. 

Sample Description 

Sixteen performance assessments at 16 school sites were selected to comprise the study 
sample. The 16 sites are identified in Exhibit 2. 
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EXHIBIT 2 



16 Round One Sites* 



SITES 



State-level assessments: 

• Arizona Assessment Reform, Manzanita High School 

• Kentucky Assessment Reform, Breckenridge Middle School 

• Maryland Assessment Reform, Walters Middle School 

• New York Regents Portfolios, Hudson High School 

• Oregon Assessment Reform, Crandall High School 

• Vermont Portfolio Assessments, Maple Leaf Middle School 

District-level assessments: 

• Harrison School District 2's Performance-Based Curriculum, McGary Elementary School (CO) 

• South Brunswick Unified School District's 6th Grade Research Performance Assessment, 
Windermere Elementary School (NJ) 

• Prince William County Public Schools' Applications Assessments, Westgate Middle School (VA) 

School-level assessments: 

• Language Arts and Math Portfolios, Ninos Bonitos Elementary School (CA) 

• Primary Learning Record, Park Elementary School (NY) 

• Rite of Passage Experience, Thoreau High School (Wl) 

National-level assessment projects: 

• New Standards Project, Ann Chester Elementary School (TX) 

• New Standards Project, Noakes Elementary School (IA) 

• Coalition of Essential Schools, Cooper Middle School (NM) 

• College Board's Pacesetter Mathematics Program, Sommerville High School (MD) 



* All schools have been assigned pseudonyms. 

Data Collection Activities and Instruments 



Because we were interested in obtaining information about the performance assessment, the 
educational context within which it was developed and implemented, and the assessment's effects at 
the local level, we collected documentary, phone interview, and site-visit data. 

Documentary Data 



Prior to and during each site visit we collected background documentary data about the 
subject assessment. The available data vrried across assessments. Types of data collected include: 
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• Descriptions of performance assessments, 

• Samples of performance assessments, 

• Policy documents about the assessments, 

• Policy documents about related education reform efforts, 

• Evaluation and research reports regarding the assessments, and 

• Newspaper reports about the assessments. 

These data were collected from state and local education officials, school staff members, 
and representatives of external groups involved in assessment reform (e.g., the New Standards 
Project). These data were collected throughout the life of the project. 

We also collected documentary data about the school sites we visite d. These data included 
reports describing each school's demographic composition, staff description, financial resources, and 
other relevant documents. 

Phone Interview Data 

Prior to each site visit, we also conducted initial telephone interviews with cognizant 
individuals in state and local education offices, the school site, and external assessment reform 
on nizations. We used an interview protocol tailored to the role of the interviewee and to the 
pertormance assessment system under investigation. 

Site Visit Data 

In the Spring of 1994 we visited the first 14 schools in our sample. In the Spring of 1995 
we revisited seven schools and added two new ones to our sample. In total, we conducted 23 site- 
visits. 

Each site visit lasted one-and-a-half to two days and was conducted by a team of two 
researchers. The researchers interviewed a number of individuals, observed classrooms, and, 
whenever possible, observed professional development sessions devoted to the development and use 
of performance assessments, administration of performance assessments, other activities related to 
the implementation of performance assessments. 

We used semi-structured interview protocols during our site visits. The protocols for both 
waves of data-collection were quite similar in structure, but wave-two protocols contained more 
probing questions about the use and effects of performance assessments on teaching and learning. 
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Case Study Data Presentation 



Each case study write-up is divided into the following four sections: 

. Section One; School Profile and Introduction . This section briefly profiles the 
organizational characteristics of the school, such as demographic data about its 
students and the community it serves. The section also presents information on the 
number and roles of the individuals interviewed and the types of observational data 
collected. 

. Section Two: Description of Performance Assessment . This section presents a 
brief history of the development and implementation of the performance assessment. 
It also presents the key characteristics of the assessment, including purposes, 
content areas assessed, scoring procedures, and technical characteristics. 

* Section Three; Context of Implementation . This section includes a summary of 
the policies and procedures followed in implementing the performance assessment, 
the resources and help available to the education agency or school personnel for 
developing and implementing the assessment, and the coordination (or lack thereof) 
between performance assessments and other tests, reforms, and organizational 
changes. 

• Section Four: The Use and Impact of Performance Assessment . This section 
describes the uses of the performance assessments by teachers and students at the 
sample school. In addition, it documents the school community's evaluation of the 
usefulness and quality of the assessments and the impact of the assessments on the 
teaching and learning processes at the school. 



Taso Study Data Interpretation 

A research design such as the one we used has strengths, but it also necessarily imposes 
certain limitations on the interpretations that can be drawn from the data. We briefly outline 
general limitations of our case study approach to data collection/ 

First, although we attempted to obtain a representative sample of performance assessments, 
we are not certain that the assessments initiated at the district and school levels are. in fact, 
representative of all district- and school -initiated performance assessments. 

Second, our information regarding the facilitators and barriers in assessment reform, 
especially at the national-, and state-levels may be less comprehensive than for those at the district- 
and school-levels. This limitation stems from the local-level emphasis of our study. We collected 
information regarding national- and state-level assessment reform from documents and general, as 



: Specific limitations are discussed in Volume I: Findings and Conclusions and Volume 
III: Technical Appendix — Research Design and Methodology. 
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opposed to detailed and probing, interviews. In addition, we did not conduct in-person interviews 
with state officials and researchers involved in national-level efforts as we did with district- and 
school-level personnel. 

Third, our findings regarding the impact of national-, state-, and district-initiated 
performance assessments are valid only for the schools included in this study; the results obtained 
for a particular school cannot be generalized to other schools involved with the same performance 
assessments. 



Finally, interviewees' opinions regarding impact of and problems with performance 
assessments signal the existence of those impacts and problems, but the absence of such opinions 
does not necessarily suggest the absence of impact of or problems with performance assessments 



O 
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Coalition of Essential Schools: 
Cooper Middle School 
April 7-8, 1994 
March 23-24, 1995 



COALITION OF ESSENTIAL SCHOOLS: 
COOPER MIDDLE SCHOOL 



Introduction 

The focus of this case study is the development and implementation of performance 
assessments at a member-school (Cooper Middle School) of the Coalition of Essential Schools 
(the Coalition). Cooper Middle School, like the other Coalition schools, develops its own 
performance assessments, based on the principles and philosophy of education expounded by the 
Coalition as a whole. 

Cooper Middle School lis located in Santa Fe, NM. The scenic beauty of the Sangre de 
Cristo mountains, the distinctive adobe architecture, and ethnic and cultural diversity fuse to lend 
a special flavor to this southwestern city. Tourists flock to revel in its attractive ambience, and 
artists have long captured its compelling colors in various mediums. Behind its attractive facade, 
however, the community is troubled. 

Community schools, according to study participants, are witnessing tensions between 
whites (“Anglos”) and Hispanics; high schools are experiencing a student drop-out rate of around 
11.4 percent; and the area is suffering from student gang violence. Furthermore, the community’s 
lack of respect for the teaching profession is a significant hurdle continually confronted by the 
public school system. It is against the backdrop of such troubles that Cooper Middle School is 
attempting reforms to better meet the needs of its students and to prepare them to become lifelong 
learners. 

Reflecting the community at large, 54 percent of Cooper Middle School's approximately 
630 students are white, 45 percent are Hispanic, and less than 1 percent are Native American. 

The school spans grades seven and eight, and its graduates advance to the two local public high 
schools (and some attend the private schools in the area). About a third of its students qualified 
for Chapter I funds in 1993-94, but the figure dropped down to 25 percent in 1994-95, due to 
redistricting and to the opening of a new middle school. The state-wide figure for Chapter I 
eligible students stood at 26 percent in 1994-95. 

Participants 

Several individuals were interviewed for this case study, as illustrated in Exhibit I. 
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EXHIBIT I 



Study Participants 



v . .. . 






• Coalition of Essential Schools Researcher 

• Principal 

• Twenty-two Teachers 

- 10 seventh grade teachers 

- 12 eighth grade teachers 

• Two eighth grade students 

• One local NEA Representative 

• One parent 

• One School Board Member 


• Principal, Cooper Middle School 

• Three teachers 

- one seventh grade 

- one eighth grade 

- one elective 

• Eight students 

- 4 seventh grade students 

- 4 eighth grade students 

• One local NEA Representative 

• Two parents 

• One School Board Member 



Observations 



In addition to conducting these interviews, in 1994 the study staff observed a student-led 
mock court session and in-class instruction, and in 1995 they observed three in-class periods. 

This report first outlines the Coalition's educational and assessment principles and then 
reviews their manifestation and impact at Cooper Middle School. 



Coalition of Essential Schools 

The Coalition of Essential Schools was established in 1984, at Brown University, as a 
school-university partnership to help “redesign” schools. As of December 1, 1993, Coalition 
members included 150 schools that are actively involved in school reform. The work of member 
schools is guided by a set of nine Common Principles, intended to help initiate a conversation 
about important issues in education and to provide a broad framework for reform activities (see 
Appendix A). 

Assessment Reform 



The sixth of the nine common principles pertains to assessment, and states that students 
should be awarded a diploma only upon successfully demonstrating — through an exhibition — 
that they have acquired the skills and knowledge central to the school's program. As the diploma 
is awarded when earned, the school's program proceeds with no strict age grading and with no 
system of credits earned' by 'time spent’ in class. The emphasis is on the students' demonstration 
that they can do important things ( The Common Principles of the Coalition of Essential Schools, 
Appendix A). 




1-2 



17 




The Coalition suggests that a process dubbed planning backwards is essential to all 
aspects of reform, including assessment reform. Planning backwards entails first developing a 
vision for the school and then taking the steps to realize that vision (ergo, “planning backwards”). 
Joe McDonald, a senior researcher at the Coalition, has enumerated the steps in planning 
backwards as follows: 

• Defining a vision. To define a vision, the school sets aside any curriculum and 
testing requirements, and generates visual images — that is, exhibitions, of what it 
expects of its graduates. Exhibitions “. . . originate as articulations of a school's 
vision — movies for a mind's eye, portraying graduates using their minds well.” 1 
These visions are “. . . concrete images of real kids” that stimulate teaching and at 
the same time function as an assessment tool. 

• Building a platform. In building a platform, the school compares what its 
students are able to do with its own ideal vision of a graduate. In comparing the 
actual performances to the ideal, the school uses its assessment system as the 
pivotal tool of reform. Based upon real versus ideal comparisons, the school 
begins to change its structures and processes to come closer, in reality, to the ideal 
vision. 

• Rewiring. This step entails overhauling the structures and process in a major 
wa y rewiring the system, so * j speak, based upon the questions and information 
emerging from the second step (building a platform). Rewiring includes steps such 
as . . . the invention of new junctures at which, for example, teachers come 
together to exchange graded papers and discuss grading standards, or at which 
parents and teachers and members of the larger community come together to share 
perspectives on achievement, or at which classes combine across grades and levels 
to discuss the same texts or problems or to get involved in some activity. The 
method centers on the act of taking the steps to invent a culture of inquiry and 
collaboration. 

Tuning. Continual tuning of the rewiring process is viewed as critical to keeping 
alive the school's commitment to self-examination and improvement. An important 
as P- -t of tuning entails extending the information network to include the 
community at large. 



Thus, as the school approaches its vision, it develops a more extensive information network that 
incorporates all stakeholders. Through this information network, the school includes the 
community in its reform efforts and benefits from the information it receives from the world 
outside its physical boundaries. 

Research and Disseminati on 



Through its newsletter, Horace, and through other publications, information from the 
Coalition's extensive research and development activities is disseminated widely to member 



'Joseph McDonald, (February, 1W2). Steps in pl anning backwards . Coalition of Essential Schools, 
Brown University, pg.1. 
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schools and to other interested audiences. For example, the Autumn 1993 list of publications 
includes titles such as Steps in Planning Backwards by Joseph McDonald and An American 
Teacher's view of British Assessment Practices , by Mu ’’shall Cohen. In addition, the Coalition 
has produced an electronic publication entitled The Exh bition Collection that contains samples of 
exhibitions developed by member schools, including samples of student work and assessment 
standards. The Coalition also holds regional and national conferences and round-table discussions 
centered around a variety of reform concerns — such as exhibitions of student work and 
pedagogical strategies. 

In addition to such dissemination efforts, the Coalition also works with RE.'Learning, a 
partnership that includes the Education Commission of the States (ECS) and all member schools, 
districts, and states. RE.'Learning provides financial and professional help for reform efforts to its 
members. 



School Context 

Cooper Middle School's assessment reform is embedded within, and is an integral 
component of, its larger systemic reform program. Cooper joined the Coalition and launched its 
comprehensive reform program in 1989, at which time the school realized that it could no longer 
operate as a “. . . mini high school and hope to meet the needs of early adolescents.” 2 Because 
of its reform efforts, the school was selected in March 1991 as the New Mexico Middle School of 
the Year. 

Through much of this time, the school essentially struggled on its own, without much 
professional or systemic support and involvement from the district, but this state-of-affairs 
changed somewhat during the 1994-95 school year. In 1994-95, the district became interested in 
the portfolio system, and Cooper is now in the position of helping one of the local High Schools 
develop portfolios for its ninth grade students. As a result of this cooperation, Cooper students 
portfolios will be accepted by the area high schools, when Cooper students advance to the high 
school level. 

Two other major district-level changes affected Cooper as well. The district acquired a 
new Superintendent, who, according to several study participants, will seek greater accountability 
from the schools. Cooper teachers, thus, already are contemplating the alterations they might 
have to make in the future to their home-grown curriculum and assessment strategies. In addition, 
Cooper's boundaries changed, requiring semi-adjustment on the parts of teachers and the Principal. 

School Mission and Reforms 

Cooper's mission statement is as follows: 

Students who graduate from Cooper Middle School will have the critical thinking 
and social skills to develop into positive, responsible, and productive citizens who 
can envision how they can contribute to the society in which they find themselves. 

They will be empowered with the confidence to be independent, resourceful, 
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committed and successful lifelong learners who will see their individuality and that 
of others in a positive light 3 

As a result of the reform process launched in 1989, the school reorganized itself into 
family groups composed of a core group of teachers and students. Through the 1993-94 academic 
year, each of the two eighth grade families were composed of six teachers and a heterogeneous 
group of students, and each of the two seventh grade families were composed of five teachers and 
a heterogeneous group of students. (All special education students were mainstreamed, excepting 
12 students .'assified in the “D” category.) The elective teachers comprised a distinct, STAR, 
family, which is responsible for teaching subjects such as music, arts, home economics, foreign 
languages, and computers to both 7th and 8th graders. 

For the 1994-95 academic year, however, the family structure was altered to form three 
teams of three teachers in the seventh grade, and two teams of three teachers and one team of six 
teachers in the eighth grade. These changes were decided upon during a school 

planning session in the Summer of 1994. Cooper staff came to believe that the extant family 
structure had not functioned well during the previous year, because teachers had formed families 
based on personal compatibilities, rather than on academic requirements. In one family, for 
example, none of the teachers had felt comfortable teaching mathematics and science, and, 
therefore, the family could not provide its students rigorous training in those two subject areas. In 
addition, teachers also believed that smaller family sizes would be more conducive to planning, 
implementing, and evaluating the family-based curriculum and assessment systems. 

The structural changes discussed above did not affect the functional features of the family. 
Each family plans its own curriculum, schedule, and student counseling services. Reportedly, 
coordination of school programs is done through a ‘'shared decision-making process” consisting of 
the principal and a representative from each family (or, regarding certain matters, the entire 
family). 



Following the Coalition philosophy, each family presents a large part of its curriculum 
materials focused on essential questions that lead students to final assessments centered on their 
abilities to use critical thinking skills in order to solve real-world problems .* The school has 
spelled out the critical thinking skills requirement into several exit competencies, which students 
are required to demonstrate upon graduation. These competencies, developed by a task force of 
teachers and parents, incorporate state and district competency requirements. The focus on exit 
competencies is provided through thematic units integrating skills, knowledge domains, 
instructional strategies, and assessment systems. (Two samples of thematic units are presented in 
Appendix B.) All thematic units and assessments are designed by teachers themselves and are 
organized around essential questions. 

The seventh grade thematic units are generally organized around the essential question 
Who am I? — and the eighth grade units are formulated around How do I affect society? Each 
family plans 4 to 10 thematic units and associated assessment techniques per year. Assessments, 
thus, are an integral and an on-going feature of curriculum and instruction. 
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In 1994-95, in addition to changing the family structure, there also was a change in the 
extent to which teachers used thematic units for content area coverage. As will be discussed 
under the Impact on Curriculum and Instruction section, teachers placed less emphasis on thematic 
units as the preferred pedagogical strategy, and this change was the most significant for 
mathematics. 

The development and implementation of assessments are described in the sections below. 

Performance Assessments 

The school fashioned performance assessments to: (a) better capture students' learning with 
regard to the essential questions and the integrated curriculum, (b) to inform the student about his 
or her progress throughout the school's program, and (c) to inform parents about their child's 
interests and progress in school. 

The school requires students to keep portfolios of their work, to undergo the Rite of 
Passage Experience (ROPE), and to answer an Open-ended Interdisciplinary Question at the 
culmination of eighth grade. These three comprehensive assessments build upon the numerous 
projects and assignments — performance tasks — students complete over the course of their two 
years at Cooper. 

It is important to note, however, that despite the use of a family-based integrated 
curriculum and assessment system, the school continues to use a traditional school report card that 
is sectioned into traditional subject areas. Students are assigned letter grades based not only upon 
their project work, essays, and reports, but also upon multiple-choice tests, and none of the 
comprehensive assessments — a complete portfolio, ROPE, and the Open-ended Interdisciplinary 
Question — is a district requirement for graduation. 

The development of project-based performance assessments and their characteristics, as 
well as the comprehensive assessments themselves, are described below. 

Development of Performance Assessments 

Reportedly, assessment development was, and continues to be, inspired by the Coalition's 
method of planning backward. Thus, outcomes are conceptualized first, and curriculum, 
instruction, and assessment tasks or assignments are fashioned around the resulting outcomes. 

That is, teachers first delineate the skills and knowledge they want their students to be able to 
demonstrate at the end of a unit and then design thematic units around those skills and knowledge 
outcomes. Elements of planning backwards were evident in teachers descriptions of the steps 
taken to plan for the 1994-95 academic year. 

During the Summer of 1994, and in subsequent teacher forums, teachers discussed the 
successes and failures of the previous year in terms of student performance. For example, they 
evaluated ROPE performances and noted that students rarely mentioned mathematics in their 
presentations. As a result, in 1994-95, Cooper teachers began taking a more direct, traditional 
approach to the teaching of mathematics. However, none of the teachers interviewed for this 
study detailed the processes of planning backwards in creating the assessment f< rmats and 
instruments, and assessments and curriculum were generally seen as being tightly interwoven. 
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Each family plans and coordinates its own thematic units, which vary in format and 
completion time. For example, the eighth grade Plaid Flamingos Family teachers work together 
on The Dawn of the Nuclear Age Unit, which lasts for about four and a half weeks (see Appendix 
B). The essential questions and skills for the unit are spelled out, as are the assignments and 
projects on which the student is to be assessed. 

Assignments and Projects 

Assignments and projects consist of essays, student productions, debates, or other events 
that are based on a student's completion of a research project. For example, during the Spring of 
1994, eighth grade students had to select and write an essay on a topic dealing with The 
Intersection of Medicine and Science (see Appendix C). Students were asked to research the 
topic, cover enough information in their reports to convey the topic to their readers, pay attention 
to the “quality and mechanics” of their writing, and use visual aids, such as graphs and charts, in 
their reports. An example of another assignment, a student production, is described in Exhibit II, 
below. 



EXHIBIT II 
Court of Law 



An eighth grade family held a mock court trial as the culminating activity of a thematic 
unit entitled Freedom and Responsibdity. 

The topic of the mock trial was, Should the United States of America ban the sale, 
manufacture, and use of cigarettes? One Judge, a prosecutor, a defense attorney, witnesses, a 
jury, a transcriber, and a court marshall, all played by eighth grade students, constituted the 
“court.” About 42 students sat in the audience and one student videotaped the proceedings. One 
at a time, the lawyers put forth well-informed arguments, produced expert testimony and 
witnesses, cross-examined one another's witnesses, and delivered closing statements. After a brief 
deliberation, the jury rendered its verdict. The court proceeding lasted about fifty minutes. 

At the conclusion of the mock trial, all students were required to produce a persuasive 
piece of writing based upon the court proceedings, to be incorporated into individual student 
portfolios. 



Scores on assignments and projects like the above are determined through scoring rubrics. 
Sometimes one teacher scores one entire assignment or project, while at other times several 
teachers score various parts of an assignment, so that grades for the traditional subject areas can 
be determined. 

Scoring Rubrics 

The design of each scoring rubric is determined by the nature of the assignments and 
projects and by the expertise of the teacher who designs it. Each scoring rubric outlines the 
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dimensions of the assignment to be graded and the score distribution or the maximum score value 
for each dimension. For example, for an essay, What is a Hero?, the scoring rubric established 
three dimensions: Writing Skills; Content; and Format (see Appendix D). 

Each major dimension was further divided into sub-dimensions: the Writing Skills 
dimension consisted of spelling, punctuation, and sentences; the Content dimension consisted of 
the exposition of common qualities of heroes and the explanation of those common qualities; and, 
finally, the Formant dimension consisted of an introduction and a conclusion. The Writing Skills 
and Content sub-dimensions were to be rated on a 5-point scale, and the Format sub-dimension 
were to be rated on a 3-point scale. The teacher-marked scoring rubrics are returned to the 
students for feedback about their performance and as an instructional tool for further work. 

Comprehensive Assessments 

In addition to the Family designed projects and assessment practices, each student is 
expected to compile some of her or his scored projects and assignments into portfolios, to 
participate in the Rite of Passage Experience (ROPE), and to answer an Open-ended 
Interdisciplinary Question. These standard school-wide assessments build on one another to 
provide a cumulative record of the student's academic work at Cooper, and are used to inform 
students and their families of student progress with regard to critical thinking and social skills. 

The teacher who is responsible for the student's portfolio and ROPE communicates the 
information to the student and to his or her family. However, as previously mentioned, none of 

these comprehensive assessments is required for graduation, and none is reflected on the student's 
report card. 

Portfolios 



All seventh and eighth grade students are required to maintain a portfolio that each edits 
during the last term of eighth grade. This portfolio must include: 

Projects or assignments that highlight the student's work in the areas of Language 
Arts, Math, Science, and History and Social Studies; 

• One piece of creative or expressive work that reveals something about the student; 

• A best piece of work from an arts class; and 

A two-to-three page essay on the student's academic, physical, and social growth 
during the year. 



Each student edits her or his portfolio during the last term of her or his eighth grade year. 
Next, each student's family of teachers examines the portfolio and assesses her or his critical 
thinking skills, interests, and areas of strengths and weaknesses, and provides the student with 
written feedback. In the seventh grade, the portfolio is not used to provide such written feedba :k. 
Although the portfolio does not receive an overall score at the end of the year, it is primarily ust \ 
for overall self-evaluation during the last nine weeks of school. 
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For the First time, at the end of the 1994*95 academic year, eighth grade students also will 
winnow their portfolios and send their exemplary portfolio pieces to their future high school. 

Rite of Passage Experience (ROPE) 

All graduating eighth graders must undergo a Rite of Passage Experience (ROPE). 

Students design and present to a panel of teachers, parents, and community members an 
audiovisual master portfolio of their experiences and growth while at Cooper. The panel acts as a 
“. . . sounding board for the students' self-reflections and self-appraisals.” 5 ROPE typically lasts 
for 15 minutes, and a record of the ROPE conference is sent to the student and to his or her 
family. (The ROPE report format, again, depends upon what the family wants to do.) 

Interdisciplinary Open-ended Question 

Students write an answer to an interdisciplinary open-ended question at the time they enter 
seventh grade, and then again during the spring semester of eighth grade. A comparison of the 
student's seventh grade essay with his or her eighth grade essay enables the student to appraise his 
or her growth over the two-year period; the student's eighth grade answer is incorporated into his 
or her final portfolio. Each family develops its own rubric to evaluate the interdisciplinary open- 
ended question. 

Resource and Professional Development Support 

Cooper participated in a fair number of professional development activities in the past few 
years. Prior to the school joining the Coalition, Cooper Middle School's Principal went to visit 
the Coalition's headquarters at Brown University to review some programs and to meet with 
researchers and program directors. The Coalition, in turn, sent researchers to help teachei with 
planning backward for curriculum, instruction, and assessments. In addition, for training and 
professional development, teachers visited schools in San Diego, Kentucky, and New York City, 
which are experimenting with developing and implementing performance assessments. The 
visiting teachers specialized in specific assessment and instructional strategies, such as portfolios 
and cooperative learning. 

By 1993-1994 the focus of profes ional development activities had shifted almost entirely 
to the process of whole-staff decision-making. Seven teachers visited other schools specifically to 
study decision-making processes. In addition, the school added five minutes of instructional time 
per period (at its own discretion). It also provided five days in-service for the year, three of 
which were used to “ action plan ” changes for overall school reform. Few teachers, thus, 
remembered receiving in-house professional development geared towards the development and use 
of performance assessment strategies. During this year, the Coalition provided information on 
assessments and related issues primarily through publications. 

During both years teachers emphasized their need for in-class training and support for the 
express purpose of designing and using assessments and thematic units. They repeatedly said that 
they wanted mentors to be working beside them to provide them with hands-on, authentic training 
and experience. Teachers noted that they often have to help themselves, and, although they are 
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quite resourceful, the process can be quite burdensome. For example, the special education 
teacher felt he did not fully understand how to use portfolios and that he required gr eater support 
in the design and utilization of what he called “true” portfolios. He did not remember any 
professional development over the past three years that explicitly centered on performance 
assessments. In fact, the teacher observed, one of the greatest impacts of the reform efforts has 
been the need for “intense professional development,” geared toward classroom practices. The 
Principal was aware of the situation and hopes to make extensive use of in-house expertise in the 
coming academic year. He plans to encourage teachers to spend time with other families in the 
school. 



In the meantime, to facilitate thematic teaching and the use of performance assessments, 
teachers generally schedule 45 minutes per day for team-planning and another 45 minutes for 
personal use. According to the Principal, teachers generally set aside their personal time on 
Tuesdays and Thursdays for parent-student-teacher conferences, on Mondays and Wednesdays for 
curriculum planning, and on Fridays for “other things.” He said that the teachers generally spend 
about two to three hours per week for assessing how their students are performing on thematic 
units. 



Cooper's professional development activities discussed above were supported by the New 
Mexico RE: Learning Partnership and the Panasonic Foundation, who provided $5,000 and $2,000 
respectively per year. Begun in 1988, the RE:Leaming partnership comprises the Coalition, the 
Education Commission of the States (ECS), and all member states, district, and schools. Its aim 
is to assist systemic reform efforts at any level of the education system. 

According to the Principal, the RE:Leaming grant in 1993-94 was used for professional 
development and for reimbursing substitute teachers at $50 per day. In 1994-95, it was used for 
professional sessions devoted to evaluating Cooper's old competencies and for developing new 
ones. The Panasonic funds in 1993-94 were used for keeping school statistics, such as student 
attendance rates, and in 1994-95, the Foundation helped pay for professional development and 
planning sessions focused on evaluating Cooper competencies. 

For obtaining such financial and in-kind support, Cooper Middle School submitted 
proposals explaining what they would do with the funding. They must identified extant needs in 
the school and how the money and the resources would help fulfill those needs. 

Interaction with Other Reforms and Tests 

Other Reforms 

As mentioned previously, the assessment reform was and is part and parcel of an overhaul 
of the school's organizational structure, curriculum, and instructional strategies. Supposedly, 
assessment techniques and results are used to drive changes in instruction and curriculum through 
the concept of planning biciwards, but most interviewees suggested that during the initial stages 
ot reform, the conceptualization and implementation of these changes were close to simultaneous. 



A major outcome of the overall reform effort was the institutionalization of a goal-setting 
process for new students called Two Weeks into Your New Experience (TWINE). During the 
TWINE conference, the individual student, his or her parents, and the teacher talk about the 
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school's program and each person's expectations for the academic year TWINE, thus, functions as 
a goal- setting mechanism for ROPE and also as a method of inviting parental participation in 
Cooper students' education. 

In 1994-95, although planning backwards as a technique of systemic change was not 
identified as such by most study participants, “exhibitions” of student performance and data from 
other assessments were used to evaluate and change the family structure and pedagogical 
strategies. 

Other Tests 

In addition to its own comprehensive and on-going performance assessments, the school 
administers, each spring, the Iowa Test of Basic Skills (ITBS) to eighth grade students, including 
all Chapter I and Special Education students. The ITBS is required by the district. According to 
the teachers and to the Principal, district-wide scores on ITBS have not increased in the past year, 
but Cooper's scores have been higher than those of other middle schools. In the 1993-1994 
testing period, Cooper students scored at the 64th national percentile in reading, 49th in math, 

50th in social studies, and eighth in science. Figures similar to those were obtained in 1992-1993. 

In 1993-94, teachers said that they did not alter their instructional strategies to prepare 
students for the ITBS, and, ostensibly, there was no coordination between the school's curricular, 
instructional, and assessment strategies (to match the multiple-choice format) and the 
administration of ITBS. 

In 1994-95, however, as will be discussed in greater detail under Curriculum and 
Instruction, teachers did change their overall pedagogy strategies for mathematics and science, 
because Cooper students' ITBS results were not satisfactory. Teachers, however, disliked the 
IT3S, as they felt that it does not have an acceptable structure for assessing the kinds of skills 
and competencies they are trying to instill in theii students. Students, too, disliked the ITBS and 
find it to be “useless.” In fact, one student said that she did not perform well on the ITBS, 
because it contained material she had not yet been exposed to in her mathematics classes. 

Impact of Assessment and Other Reforms 

No formal studies have evaluated the impact of the myriad reforms at Cooper. However, 
the school community is taking note in detail of the complexity of the effects upon itself of such 
changes. Such effects are discussed below. 

Impact on Teachers 

Although Cooper teachers support the reform activities and outcomes, the “transition has 
been difficult,” said the Principal. Teachers shoulder the demanding tasks of fashioning their own 
thematic units and assessments to judge student achie' ement. For them, this i as raised, rather 
than resolved, several issues. (Many teachers, especially in math, continued to use traditional, 
multiple-choice tests in addition to performance assessments for the purposes of evaluating student 
performance and assigning traditional letter grades to their students.) 
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Quality and Standards . For many teachers, among the other main points to be debated 
are assessment quality and standards. In 1993-94, some of the teachers were worried that the 
quality of their assessments, i.e., reliability and validity, might be questionable; by 1994-95, this 
worry extended to the appropriateness of their teaching methods as well. In 1994-95, teachers 
also observed that it had become increasingly difficult for many of them to be generalists and to 
rotate the teaching of several subject areas. 

One teacher verbalized the difficulty as feeling ”. . . very frustrated. “ He said that 
teacher needed to explore other models of thematic teaching and special education inclusion 
practices. Through the thematic ui >ts they designed and the rubrics they generated and refined 
every year, they maintained certain expectations of ar.u objectives for their students, but the larger 
educational objectives remained obscure. (In 1993-94, for example, some teachers were not fully 
aware of the school's defined competencies, and, by 1994-95, new competencies were being 
developed.) The NEA teacher representative stressed that outcomes needed to be clearly defined 
and sequenced, if the assessments were to be useful. 

Report Card . Another major dilemma teachers continued to face is that the report card 
they must use is organized by traditional discipline areas, and certain skills and competencies 
must be covered within those subject areas. Assignment of subject area letter grades based on 
interdisciplinary thematic units is, therefore, quite challenging. As one teacher said, teachers are 
doing “two dances — [the] innovative and [the] required dance,” which was a frustrating 
situation. (As mentioned above, part of a student's grade in the traditional subject areas also 
derived from these assessments.) 

In order to assess and report the attainment of Cooper competencies in a uniform format, 
the Cooper staff worked on designing a rubric-style report card during 1994-95. The Principal 
hoped to use this report card (in addition to the traditional report card) at the end of the 1994-95 
academic year. 

Professional Support . Teachers also talked about needing information on how to score 
entire projects and wanting on-going technical assistance, more money, and more time to devise 
assessment strategies and thematic units. They could not really discuss assessment issues without 
addressing instructional concerns, and, hence, stressed that the drawbacks of project-oriented 
instruction and assessment were that therr was very little time to “refine” units and assessments. 
Teachers have about 45 minutes per day for planning, and that is too little to discuss the benefits 
and drawbacks of the new instructional and assessment techniques. 

Impact on Curriculum and Instruction 

Cooper teachers regularly used a variety of instructional techniques, ranging from 
engaging students in Socratic discussions, cooperative learning, debates, and lectures accompanied 
by audiovisuals. The principal maintained that the teachers use more varied instructional 
strategies now than they did before the reforms were mobilized Positive educational changes, 
however, both teachers and principal agreed, cannot be driven solely by revamping assessments; 
issues in curriculum and instruction must also be considered. 

In 1993-94, some teachers were concerned that thematic units might interfere with 
sufficient coverage of traditional discipline areas. According to the NEA representative, student 
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choice in determining thematic assignment topics precluded comprehensive coverage of content 
for every student. By 1994-95, this concern was particularly salient for comprehensive coverage 
of mathematics, science, and history. 

In the Summer of 1994, using ROPE as an informational tool, teachers identified what 
they perceived to be weaknesses in their mathematics curriculum. They were startled to discover 
that students hardly ever mentioned mathematics in their culminating performances, ROPE, either 
as something they had learned or as something with which they had experienced difficulty. 
Teachers also were disappointed in their students' 1TBS mathematics scores. They came to the 
conclusion that teaching mathematics mostly through applied problems and thematic units were 
not adequate methods of instilling in their students a good understanding of the discipline. These 
teachers felt that their methods had not been successful either because the methods had been 
developmentally inappropriate for their students, or because they did not possess sufficient 
mastery in using these methods. In either case, teachers had experienced “constant frustration” of 
teaching mathematics as an applied science, said one teacher. 

Cooper teachers, thus, decided to employ a more direct skills-building, “old fashioned” 
approach to the teaching of mathematics. During 1994-95, therefore, teachers placed less 
emphasis on teaching mathematics through thematic units and a little more stress on “pure” 
mathematics. Every student, thus, was assigned to a fifty minute mathematics class. One eight 
grade family also decided to group its students by ability levels for its mathematics classes. In 
addition, to further understand the issues involved in mathematics curriculum, a school committee 
was established to study the NCTM standards to further refine the school's mathematics program. 

Cooper teachers also refined other disciplinary areas as well as thematic units. During 
1994-95, in fact, teachers identified all disciplines by their names, and they also tried to 
emphasize traditional teaching methods. According to one teacher, part of the pressure came from 
the parents, who asked, “What class is this?” (It also was evident from the parent-teachers 
conferences that parents felt that their children were not being taught sufficient amounts of 
mathematics and science and that these areas had to be strengthened.) Thus, for example, 
American History was presented in a more chronological fashion, and science was infused even 
more into thematic units and was taught as a separate subject. 

Teachers of elective subjects said they had always used performance assessments, but that 
they now had written standards and benchmarks (but no scoring rubrics) to judge student 
performance. The electives teachers also said that assessment reform was concentrated primarily 
in the “core” classes, as there was not much need for such reform in their classes. The 
technology teacher said she supports reforms in the “core” classes by requiring students to play 
electronic games that promote inductive and deductive reasoning and problem-solving skills. 

Impact on Students 

In 1993-94, according to the two students interviewed, thematic units were enjoyable and 
confusing. The two mentioned that students were uncertain about which teacher taught what 
subject, a factor that caused them some distress. For example, one student mentioned that he 
would have liked more help in mathematics, because he was not proficient in the subject, but that 
he did not know who to approach for help. (He aspired to study civil engineering in college, but 
needed a rigorous math background do be able to do so.) 
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Another student said that she did not like the thematic units because they did not ’’teach 
subjects.” (Because of her elementary school training, she was accustomed to being exposed to 
extensively defined subject areas.) In addition, both students were worried about gaining content 
area knowledge, because the report card, the basis for graduation, was organized by subject area. 

A related disgruntlement these students harbored was that they did not understand how 
subjects were interrelated within thematic units, or what, exactly, they were expected to know at 
the end of those units. Students articulated the desire for . . more organization of themes — 
everything is mixed up in themes; we don't know where we are.” One student claimed that he 
was “lost” when they covered the essential question, “ What does it mean to be human?” 

Students also complained that they found thematic units burdensome, as they were asked 
to complete research-based assignments, even though they frequently were not given a 
bibliography or resources, and despite the fact that the library did not contain much research 
material. These students also did not evince strong feelings about portfolios, because it was 
not a big thing,” according to one. That student said that it was only “partly kids' choice” as to 
what completed assignments were to be included in the portfolios. One student, in fact, kept 
using the word “miscommunication” about the process of education at Cooper. Paradoxically, 
both students did not mind creative assignments, and one said that he especially liked debating. 

By 1994-95, many of these difficulties had been allayed, primarily as a result of clearer 
identification of traditional subject areas and due tn the changes in the family structure. Most 
students reported that their classes were “pretty separated,” and they knew when they were 
studying mathematics, science, English, and other subjects. Their reactions to thematic units also 
were more favorable. They said that they enjoyed doing “hands-on” assignments, such as 
monitoring the stock market for mathematics and making “dragsters” for science projects. They 
also said that they liked keeping portfolios, because they can revisit the work they had done. 

These students also enjoyed group work when they did not have to finish other students' work, 
which did happen from time to time. 

Students' reactions to the grading system showed a preference for the traditional letter- 
based grading system, perhaps because that is the basis for Cooper's report card. In 1993-94, 
although students appeared not to be overly concerned with the grading system, because, as one 
said, it was “fair, more or less,” they mentioned wanting more feedback and a better 
understanding of the grading criteria. They also said the grades seemed to them based on 
accumulating “things,” rather than on the “quality” of what they prod>’^. In 1994-95, students 
were clearer about the use of scoring rubrics for setting performance standards and for scoring 
purposes. Nonetheless, one seventh grade girl said that she liked the traditional letter-grade 
system, because an “! A ' shows me that I'm doing really good.” 

In 1993-94, both the principal and the teachers reported that as a result of the classroom 
teaching and learning strategies employed at Cooper, their students were developing powerful 
critical thinking skills, good writing skills, the motivation to achieve high standards, and the 
ability to work together groups. The eighth grade teachers observed that their students were much 
mi re challenged through the project work than through text-book based work, and were focusing 
on “processing information.” Interestingly, teachers from one family asserted that the one direct 
result of assessment was that scoring rubrics helped students understand the grading criteria, so 
that they no longer disputed their grades. 
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At the end of the 1993-94 academic year, however, teachers also noted that students were 
not proficient enough in mathematics and in science. In addition, because of the team structure, 
teacher rotation of subject areas, and the use of thematic units, students often were confused about 
which teacher taught what subject and about the connections between thematic units and subject 
areas. As a result of this realization, the team structure and pedagogical strategies underwent 
significant changes. These changes are discussed in the sections below. 

Special Education Students . The special education teacher asserted that performance 
assessments was “wonderful” for special education students, but such assessments, especially 
exhibitions and portfolios, tended to reward more “overt and gregarious children. He said that 
children who are emotionally and behaviorally disturbed did especially well on performance 
assessments because they relied on verbal skills to express themselves. The special education 
teacher also believed, however, that other types of assessments that place less emphasis on “show” 
must be devised to capture the performance of students who are shy. 

Impact on Parents and Community 

The school is regarded as a leader in assessment reform and enjoys a good reputation 
within th- community. In 1993-94, according to a school board member, the board was “very 
pleased and supportive of it [the school].” In fact, the Board v:~ trying to encourage other 
schools to reform their instructional and assessment methods. However, the board member also 
was concerned about the lack of specific standards and the communication of those standards to 
students. (He, too, was not aware of the Cooper competencies.) 

The board member's personal concern about his daughter who attended the school 
involved projects and homework. He did not “see a real structure to her projects,” and he felt 
that students were assigned “little homework.” Hence, in his opinion, the right objectives were in 
place, but not the right processes and outcomes. 

In 1994-95, the school board member also had a similarly high opinion of Cooper, but he 
had no children enrolled in the school. His information, thus, was more pertinent to the district as 
a whole. He was concerned about the district's ability to attract quality staff, as the district's 
financial incentives were so low. He also expressed the need lor establishing a rigorous district- 
wide accountability system, as he was distressed about the fact that the district scored on the 
lower end of ITBS. Although he expressed great enthusiasm for using portfolios as means of 
giving feedback to students, he did not want them to be converted into a “machine-like process.” 

In both years, all parent participants were supportive of Cooper's philosophy of education 
and teachers use of project-based work, but two parents also expressed concerns over children's 
exposure to content areas. “Method for method sake is meaningless. Transferral of content 
information is very important,” said one parent. 

I n jgq 3 .q 4 ) the parent interviewee was an active member of the school community. At 
Cooper, she coached a group of studem for an academic competition entitled “Odyssey of the 
Mind.” Although she was appreciative of the hands-on work and creativity demands of project- 
oriented work at Cooper, she was not certain that her son liked the work very much. She 
mentioned, too, that ROPE is a great idea, but she is not so sure that it means as much as it is 
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supposed to mean to the students. She, herself, was concerned about what she perceived to be a 
weak scoring system, and she wanted a more rigorous one in place. 

In 1994-95, while one parent, the President of the Parent Teacher Association, spoke very 
highly of the school, the other parent felt that Cooper standards must be raised and learning 
outcomes must be defined. (He had not yet seen Cooper'? new competencies.) Both parent were 
supportive of the plans they said the new district Superintendent has to institute greater 
accountability measures for the school system. 



Future Plans 

In the near future, Cooper staff plans to address a number of outstanding and important 
issues. It will (a) devise a better inclusion model for special education students, (b) require an 

educational plan for every student, and (c) define a better mode of transition from middle to high 
school. 



Cooper teachers currently feel that Cooper students experience more difficulties adjusting 
to high school classes than do students from other middle schools, because they become 
accustomed to project work and performance assessments. The changes in their approach to 
teaching subject areas, they hope, will help the future cohorts in adjusting to the academic 
demands at the high school level. 

The school also will finalize new Ccoper competencies and better assessment and 
measurement of student outcomes. The new competencies will be established within the state's 
curricular frameworks entitled, Standards of Excellence. In the meantime, the school has 
developed a Cooper Competency form that is intended to function as a narrative report card. 
Students will eventually be rated on Cooper Competencies at the end of seventh and eighth 
grades. The Principal also hopes eventually to use the form for advancing or detaining students. 
In the meantime, the district's new Superintendent may also require changes in assessments and 
curricular frameworks. 

The school administration also would like to move from time-bound learning to mastery- 
based education in order for students to determine their own pace in mastering curriculum units. 
However, standards for competencies and performances have not as yet been fully articulated, and 
faculty members are experiencing conflicts over how to meet all the currently established 
standards. They feel that they cannot provide variable amounts of time to students to attain 
mastery of the various units. 

All of the above plans will be subject to the scrutiny of the new district superintendent. 
The common wisdom is that the new superintendent will standardize the curriculum and seek 
more accountability. It is the hope of the Cooper community that the district will spell out the 
required competencies, but that it does not require all schools to use specific textbooks and lesson 
plans. Cooper Principal also hopes that Cooper will be used as a model school for reform 
activities in the district. 




1-16 



31 



Conclusions 



Cooper is steeped in the philosophy of school reform and is earnestly attempting to change 
the central features of education: teaching and learning. The school has taken great pains to 
reorganize itself into smaller student and faculty groups (families) and is grappling with issues in 
the areas of curriculum, instruction, and assessment. The faculty uses the information from 
student performance to judge the rigor of its pedagogical approach and philosophy. Problems, 
however, remain. 

There appears to be little central coordination in the development and implementation of 
thematic units and the accompanying assessment system. In addition, less than needed 
professional development in designing curriculum, assessment, and instructional strategies means 
that teachers are left to their own devices, which adds work to already overloaded schedules. 
Hence, several features of the assessment system are uncoordinated and without much quality 
control. Teachers tend to use “existence- proofs” (e.g., the presence of certain ideas, 
characteristics, and so on) as grading criteria; and standards for student outcomes have not been 
fully defined. 

Coupling interdisciplinary instruction and assessment with a traditional content area report 
card also is a cause for concern. Both teachers and students worry about content coverage and 
about how to infer performance in a subject area from an interdisciplinary assignment. 

Fundamentally, the issue of quality in assessment reform centers around the resources 
(basically information and time) provided to teachers for continuing the work of reform (without 
causing them to become confused about the “two dances” they are expected to perform). 

Cooper staff is aware of these issues and concerns, and, in 1994-95, began to address them 
with changes to its family structure and pedagogy. The questions that led instructors to change 
their approach to teaching, however, remain unanswered. 
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APPENDIX A 



The Common Principles of the 
Coalition of Essential Schools 



APPENDIX B 



Sample Thematic Unit 
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MO ON e A Z ER S 

WESTERN. THEM A TIC. UN IT £ j 

OVERVIEW 

STUDENTS WILL DEMONSTRATE THEIR UNDERSTANDING OF CON? 
IN THE DEVELOPMENT OF NEW MEXICO DURING THE 19TH CENTURA 

ESSENTIAL QUESTION 

HOW DID CONFLICT SHAPE NEW MEXICO'? 

WHY DID PEOPLE COME TO NEW MEXICO? 

WHAT CHANGES RESULTED FROM THE CONFLICT IN NEW MEXICO? 
WHAT IMPACT DID THE OLD SANTA FE TRAIL HAVE? 

THEME 

19TH CENTURY NEW' MEXICO 



ESSENTIAL SKILLS 

The student will be introduced to the following skills: 



writing a paper and delivering a narrative 1131c 
interpreting and relaying verbal and nonverbal 
direct ions / message sIJA6 

asking Questions to elicit specific informal lcn/opinions II A 7 
use a variety of sources to distinguish relevant and irrlevan 
inf or ma tier. 1 A1 a 

recognize social, historical political, cultural and enviror.mei 
relationships lAob 

reccnize the impact that the events of the past have *-n the 
future 1 0 A 5 d 

develop sense of chronology of events lAic 
acknowledge sources and references through use cf 
bibliographies. IB 

•. ompare and contrast various kinds of information NA 1 
write a paper and deliver an oral presentation which is cle 
.. onsise, and interesting m each cf the following modes 
informative, narrative, analytical 1131a, c, d""* 

Write using standard grammar, spelling , punctuation 
editing and proof reading work I1B2 

emew respect tor self, adults, peers, and property nr 
Generate ideas and formulate hypothesis to predict outrer 

I A ? 



Perform all four operations with real numbers and 
tc sc. vc math prcolcsm I A 4a 

Identity mas, propaganda anc faulty .ogic and dist. 
between fact and opinion II Ab 



formulas 

guish 
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■3FST C0?Y AVAILABLE 



MOON G A ZERS 

WESTER M THEM A TIC UNIT 

BODY OF K N 0 W L ED G E/ PE D AGOG Y 

JOURNALS 

MONOLOGS 

POETRY 

ISOLATION OF NEW MEXICO 

SANTA EE TRAIL 

METHODS OF COMMUNICATION 

SOLVING PERCENTAGE PROBLEMS - BLE «-- ir -VES 

USING PERCENTS TO D1SCR13* AMD .~AL_.!CT r ^or.bLE - - * ■ L - 

HOW PERCENTS PERSUADE US 

note-taking skills 

WRITING BIBLIOGRAPHIES 

[V C C £ v - 0 p fc ; v" A T 

C 0 M P A R A. T I V E W R I T I M G 

V 0 C A B U L A R Y DEVELOP M E N T 

CUTLING FORMAT AND SHILLS 

RESEARCH 

ORAL PRESENTATION 
M A P P 1 N G 
T I M E L I N E S 

j e M T o s 

DICHOS 



benchmark outcomes 

E N G L 1 S H 
SOCIAL STUDIES 



Note C arc 



M a t h 

C 0 M 1,1 U MICA T 1 0 N I 



Essay, Outline, Bibliography 
Create Individual bock 
Note Cards, Fluoric,. Picture, Map. 
Timeline 

Estimating distances, 'Io-Scale Uravur.gs 
M o n o 1 o g s , Journals, Poetry. Li u 1 1 i - M e d 

Presentations 

assessment 

The family will pick one , ,l 

1 PRODUCTION Students will genera*.? *heir materi 

iwr. production :cr "The Santa i-e Irai. , h r 

2 INDIVIDUAL BOOK Earn stucent will write hi-.-h.r 

- u n t c ot their iamily arriving ir. NM k 

v" OLD SANTA FE TRAIL JO R N A L Each sy.»cent .... r^-;- 

» ,',} an 01 c. Santa ;• e I rail Journal ai.d . i • *■ 

tr:nv ,. a.or.g -ne Old saitta 1- I rail 



BEST COPY AVAILABLE 
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THE DRUIN OF THE NUCLERR ROE 



Leuel: 



Eighth grade 



Oueruieui: Students will study the technology, history 

and issues surrounding the deuelopment of 
nuclear energy and participate in a debate. 

Theme: Nuclear Energy (a 4 1 /2 week thematic unit) 



Essential Questions: 

1. Was the U.S. justified in using the Atomic Bomb on Japan? 

2. Should the gouemment continue to support the design and 
manufacture of nuclear weapons? 

3. How safe is nuclear energy? 

4. How was the U.S. gouemment justified in using uninformed citizens 
for radiation testing during the Cold UJar? 



Bodies of Knowledge: 



Science 



Social Studies 



Mathematics 
Language Arts 



Nuclear reactions 

Genetics and mutations 

Growth and Human Deuelopment 

History and Impact of nuclear discouery 

Effects on foreign policy 

World War II; the Cold War 

Probability 

Equations 

Headings- fiction, nonfiction 
Hesearch 

Historical Fiction, writing and reading 
Public speaking/Debate 



Vocabulary 

Genetics, radiation, fission, fusion, heredity, radioactiuity, atom, 
atomic, nuc>ear, the Cold War, McCarthyism, propaganda, explosion, 
implosion, totalitarianism, dictator, racism, appeasement, facism 
Marxism, Nazism, isolationism 
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D.O.ftLA 'D11ATM 



STUDENT NAME 

DEBATE RESOLUTION: 



(5 pts.) The student turns in a com alete scnpt c t her/his 3 minute 
debate before performine M * debate . 

.(20 pts.) The Debate Topic is clearly understood. Relevant terms are 

defined (If Trst positive or first negative, there is a brief history 
given.) 

.(20 pts.) The positive's or negative s main idea is made clearly and 

the debater demonstrates convincingly that his / her position is 
valid and necessary. 

_(5 pts.) Each debater uses at least one quote which supports the 
argument. Source of quote must be noted. 

_(1 0 pts.) All factual information is given a credible source. 

_(10 pts.) Charts or graphs effectively display useful and 
meaningful information. 

_(10 pts.) Style and delivery of presentation. 

_(10 pts.) Effective use of props (Bonus pts.) 

REBUTTAL 



(10 pts.) The rebuttal is clear, well organized and well thought out. 

(10 pts.) The rebuttal effectively points out weak points in the 

opponents argument and/or successfully strengthens weak 
points in the debater s argument. 



(lOOpts.) TOTAL 
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The Intersection of Medicine 
and Science Project 
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Modem Developments in Medicine and Science 

Due: Friday, April 15— 100 points 

THE ASSIGNMENT: 

Select a topic from some point at which science and medicine intersect (their 
"nexus"). This topic should be of interest to you personally. Research the subject 
and write a report about what you have learned about this subject. 

In reporting on your topic, include as much specific information as possible about 
how it affects the human body and tell which systems and organs are affected. 

Include also information about prognosis for the future and possible breakthroughs 
in treatment. If you are reporting about such controversial medical issues as the 
abortion pill or AIDS explain why this subject is controversial and what the sides of 
the controversy are. 

Included with your report must be some sort of visual aid that helps "fill out" 
your report for those of us who need visual stimulus in order to leant For 
example, if you are reporting on AIDS you could display through pictures or 
drawings how the virus harms the white blood cells, or you could chart the 
sequence of how Cancer spreads through tissues. 

Here are a few suggestions if you can't think of anything: 

1 There are always articles m TIME, .NEWSWEEK, and PEOPLE magazines and m the newspapers 
about breakthroughs in medical research. Find at least 3 articles that deal with research about one 
disease or problem, read them, and write a brief summary of the articles, relating the information to 
what you are learning about the human body and its systems. Include the articles with the rest of your 
project 

2. There is a constant flow of information about the AIDS virus. Find out more about how the virus 
works in the body. Find photos or drawings of HIV. Display m pictures how the virus harms the white 
blood cells. Chart a sequence of what happens to the body's immunity to other diseases Gve some 
information about the latest m HIV research and list ways in which AIDS is and is not spread. 

3 . There are many different kinds of acne medications and skin lotions for sale. Most of the information 
that you get about these products is from newspaper or magazine ads, and radio and TV commercials 
Before you buy you should evaluate a product's ad da* ns. Do an evaluation by following these steps: 

( 1 ) Find and cut out a newspaper of magazine ad for an vcne medication or skm cream or lotion. (2) Wnte 
the name of the product. (3) Read the ad carefully, then make a list of questions you would like to have 
answered before you decide whether or not to believe what the ad claims. (4) What reasons might you 
have for questioning the claims in the ad? (3) What in the ad made you want to buy the product? (6) 
What in the ad seemed not to be completely truthful? (7) Would you buy and use the product in the ad? 
Explain your answer. (8) Design your own ad for a sttn care product that effectively "sells" the product 

4 . On your own research briefly, and wnte a report on some othernewsfrorr, the nexus of science and 
mediane. Eg. report on Bo Jackson's hip replacement or other sports mechane stones; cancer research 
and treatment like the use of Shark Cartilage; the necessity of vitamin and mineral supplements in the 
diet; the dangers of skm cancer as a result of exposure to the sun; the hazards of dmg/ alcohol abuse to 
the body; etc... 

You will be graded on: 1 . Evidence of research (be sure to cite your sourrc- - 

minimum of two) 

2. Quality and mechanics of your report -spalling 

grammar, punctuation, neatness of presentation, etc.. 

3. Thoroughness of the report-do you cover enough of 
the information to make the report of use and interest tc 
others who may be curious about the subject 

4 Visual aid(s)-graphs, charts, pictures, drawings etc.. 



Primary Learning Record: 
Park Elementary School 
May 5-6, 1994 
April 3-4, 1995 



43 



PARK 



PRIMARY LEARNING RECORD: 
PARK ELEMENTARY SCHOOL 



Introduction 



This case study explores the integration of a new method of assessment with a well- 
developed and well-articulated educational program and philosophy with which it is compatible. 
Educators at New York City's Park Elementary have long provided a child-centered educational 
program, and for the past few years, several of them have employed the British-developed 
Primary Language Record, and. more recently, the Primary Learning Record, to help support that 
program. 

Park Elementary is a magnet school in New York City’s District Four. Founded 20 years 
ago. the school’s guiding philosophy is to provide a child-centered education. The school has 
waivers from some district mandates, including one that allows teachers to send home narrative 
reports in lieu of standard report cards. During the 1993-94 school year. Park Elementary served 
about 250 students, pre-kindergarten through grade six. These students were 18 percent white. 36 
percent African-American. 42 percent Hispanic, and 4 percent Asian. (The demographics of the 
student body were little changed for the 1994-95 school year.) The Park Elementary staff 
includes primarily experienced teachers: about two-thirds of the staff of 14 have been at the 
school for 15 years or more, and several teachers have been at the school since its founding. All 
regular classroom teachers teach mixed-grade classes (e.g., K.-1, 3-4). and students typically have 
the same teacher two y ears in a row. 

The Primary Language Record ( PER) provides teachers with a structured method of 
tracking young children* >• development of language skills and planning individualized instruction 
to meet students' language and literacy needs. The PLR was first introduced in New York City in 
the early 1990s. The PLR has since spawned the Primary L.earning Record (PLeR), which uses 
the same approach to monitor and record children's intellectual development in all subject areas. 
Implementation of first the PLR and beginning in the 1994-95 school year, the PLeR in about 62 
New York city schools is supported by three organizations — the Center for Collaborative 
Education, the Center for Educational Options, and the Elementary Teachers Network — which 
together comprise the New York City Assessment Network. 

I his case study is drawn from two visits to Park Elementary, the first in May 1994 and 
the second in April 1995. Between the two visits, teachers had switched from using the PLR to 
the PLeR. However, their reactions to and comments about the two closely related assessments 
are quite similar. Thus, throughout this case study, the combined term. "PLR/PLeR” will be used 
to designate facts or opinions applicable to both versions of the assessment. 



Participants 

Several individuals associated with Park Elementary and the Center for Collaborative 
Education (the organization assisting Park Elementary teachers in the implementation of the 
PI.R'Pl.oR) were interviewed in Spring 1994 and Spring 1995: the roles of these individuals are 
identified in Exhibit I. 

Il-I 
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Observations 



In the Spring of 1994. lessons were observed in two classrooms (preK-K and 5-6) and 
samples of completed PLR forms were collected. In Spring 1995. lessons were observed in two 
classrooms (preK-K and K-l). 



EXHIBIT I 

Study Participants 



1993-94 


1994-95 


• Teacher Consultant at the Center for 
Collaborative Education 

• Teacher Director 

• Four Teachers (grades preK-K. 3-4. 5-6. 
and special education resource); three of 
the four teachers use the PLR 

• One parent 


• Teacher Consultant at the Center for 
Collaborative Education 

• Three teachers (grades preK-K. K-l. and 
special education resource); all three 
teachers use the PLeR 

• One parent 



School Context and the Primary Language and Learning Records 

Park Elementary is housed in an educational complex in East Harlem, which also houses 
Park Elementary's associated high school (grades seven through twelve) and a neighborhood 
junior high school. However, the wing that houses Park Elemental feels very much like an 
elementary school. The hallways are lined with children's work — paintings and drawings, 
sculpture and crafts, stories and discoveries. Children seem to feel comfortable interacting with 
all of the school's teachers, calling all staff members by their first names. 

Families must apply to have their children admitted to Park Elementary; in order to apply, 
the child and family members must first spend several hours visiting the school. Admissions are 
then made by lottery, w ith the exception of children of staff members and siblings of other Park 
Elementary students, who are automatically admitted. 

The Center for Collaborative Education and the New York Citv Assessment Network 



Park Elementary is one of 1 1 schools participating in the Center for Collaborative 
Education’s (CCE). Elementary School Assessment Project (ESAP). COE a network of small 
alternative schools ; n New York City, initiated ESAP in the early 1990s. I he mission of ESAP is 
"to develop the use of qualitative assessment techniques for instructing and evaluating children - :; 
learning in the New York City public school system. . . . using highly detailed observations and 
descriptions to capture the richness of children’s learning". 



VS allace. V. (I l > l >'. \tav ) IJemenhin School i\\e.\smenl Protect ’ Review of Year One. IW2-V.Y New 
York: C enter for Collaborative I (location, p I 
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In 1991, the CCE joined with two other New York City organizations, the Center for 
Educational Options and the Elementary Teachers Network, to establish the New York City 
Assessment Network (NYAN). The three organizations formed NYAN because of their shared 
commitment to introducing to their teacher colleagues an assessment technique called the Primary 
Language Record (PLR). a tool designed to help teachers acquire and use those techniques 
espoused in the ESAP mission. In the Fall of 1994, NYAN and its participating schools shifted 
from the PLR to the PLeR. All together, the three NYAN organizations are supporting 
approximately 300 to 400 teachers in about 62 schools in New York City as they implement the 
PLR/PLeR. ' 

The Primary Language Record and the Primary Learning Record 

As designed, the PLR is intended to accomplish two major purposes: (1) to help the 
child’s classroom teacher understand the child's learning style — focusing specifically upon 
language development — in order to tailor an educational program appropriate for the child; and 
(2) to help the teacher communicate his or her understanding of the child to the child’s next 
teacher. The PLeR serves the same purposes and incorporates the same components as the PLR. 
but it is intended for use by teachers to monitor children's development in all subject areas, not 
just language. 

Teachers at Park Elementary use the PLR/PLeR to help them better understand their 
students' learning styles, literacy progress, and other intellectual development and to plan 
instructional activities to meet the needs of individual students. Teachers do not use the 
PLR/PLeR to assign grades or to rank children on their progress. Thus, as the PLR/PLeR is 
implemented at Park Elementary, the first purpose identified above is more important than the 
second. 

Development of the Primary Language Record and the Primary Learning Record 

Doth the PLR and the PLeR were developed by British educators associated with the Inner 
London Education Authority (ILEA) and led by the Centre for Language in Primary Education. 
Starting in 1985. these educators identified a need in their schools and classrooms of gaining a 
better sense of their students' learning strategies and their emerging literacy; they identified this 
need in part because of the growing number of children in ILEA schools for whom English was 
not a first language. Piloted in 50 schools during the 1986-87 school year, the PLR was made 
available in September 1988 to all British schools interested in using it. 

As developed, the record aims to satisfy both record keeping purposes and instructional 
purposes, including: 

• Informing teachers who do not yet know the child of his or her progress: 

• Informing the principal about the child's progress: 

• Providing parents with information about the child's progress: and 

• Supporting and informing teaching in the classroom and providing teachers with a 
framework for teaching language and literacy . 
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The last objective is the one of primary importance to NYAN. CCE, and teachers at Park 
Elementary as they use the PLR and the PLeR (also developed by ILEA educators). 

Introduction of the Primary Language Record and Primary Learning Record to New 

York City and Park Elementary 

In the early 1990s. several of the British educators involved in the development of the 
PLR spoke in New York City about their experiences with it. Representatives from all three 
NYAN member organizations attended the presentation and became interested in introducing the 
PLR to the teachers they worked with. One Park Elementary teacher attended the presentation as 
well, and she began experimenting with the PLR during the 1991-92 school year. Other teachers 
at Park Elementary did not begin using the PLR until 1992-93. the year the Center for 
Collaborative Education's Elementary School Assessment Project began. Teachers at Park 
Elementary are not required to use the PLR or to participate in ESAP, but about half of the 
school’s teachers do so voluntarily. 

In the Fall of 1994. CCE and all Park Elementary teachers involved in ESAP switched 
from using the PLR to the PLeR. The switch to the PLeR was motivated by feeling among 
teachers that it was more natural to observe and record children's learning in all areas, not just in 
language and literacy development. All New York City teachers collaborating with the CCE' 
switched to the PLeR at this time. 

The Role of NYAN Teacher Consultants 



Each of the three organizations involved in NYAN — the Center for Collaborative 
Education, the Center for Educational Options, and the Elementary Teachers Network — has at 
least one teacher consultant on its staff. This individual is responsible for helping teachers at its 
member schools become comfortable with the PLR and use it successfully. These teacher 
consultants foster implementation of the PLR in several wav s, by: 

• Convening group discussions w ith teachers to speak of specific children's learning 
sty les and how instructional and curricular approaches can be tailored for children; 

• Conducting in-depth observations of individual children to share insights about the 
children's learning with their teachers; and 

• Helping teachers find ways to adapt the PER so that it fits . .urally into their 
teaching sty le and becomes an asset. 

CCEs teacher consultant works with six teachers at Park Elementary — about half of the 
Park Elementary staff. (As a former Park Elementary kindergarten teacher, the teacher consultant 
is not a stranger to Park teachers. Additionally. CCE! occupies as its office one large room in the 
same educational complex in which Park Elementary is located: thus CCE. staff are readily 
accessible to Park Elementary teachers.) During 1992-93. the CCE teacher consultant also worked 
with 33 teachers at nine other schools; she typically visited schools about once a week. By 1994- 
95. despite the change from the PLR to the PLeR. teachers had grown more accustomed to and 
adept with the PLR. and. consequently, the teacher consultant was able to reduce the frequency of 
her \isits to schools participating in I SAP. During the 1994-95 school year, she visited schools 
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about once a month. All teachers — both at Park Elementary and other CCE member schools — 
participated voluntarily with CCE and the PLR/PLeR. 

Characteristics of the PLR and PLeR 

The PLR was developed primarily to help teachers develop the language and literacy skills 
of children in the primary grades. However, the PLR can be used with older children as well, and 
Park teachers also used it in other subject areas (thus demonstrating the viability of and paving the 
way for the Primary Learning Record). 

The Primary’ Language Record 

The developers of the PLR recognized that teachers are not the only indiv iduals in 
children's lives who have insights into their language development. Thus, the PLR process is 
designed specifically to draw upon multiple perspectives of a child's development of language 
skills. To be as comprehensive as possible in its record, the PLR process: 

• Involves parents in conferences with the teacher to comment on the child's 
language development at home: 

• Records the child's own evaluation of his or her progress: and 

• Includes information about a child's language development in his or her first 
language, if that language is not English. 

The PLR consists of two types of record forms, the Primary Language Record form and 
the Observations and Samples form: 

• The Primary Language Record Form has sections for the teacher to record in the 
fall and spring of the school year. In the fall, the teacher records (a) information 
about the languages spoken and read by the child and the child's medical history, 
(b) notes from a conference with the child's parents about the child's language 
development, and (c) notes from a conference with the child about his or her 
language skills: and in the spring, the teacher records cumulative observations 
about (d) the child's listening and talking practices and development, (e) the 
child's reading development, (f) the child's writing development, (g) notes on a 
follow-up conference with the child's parents, including the parents' response to 
the teacher's comments on parts d. e. and f, (h) notes from a follow-up conference 

>ith the child, and (i) information for the child's next classroom teacher. 

• The Observations and Samples Form, on which the teacher notes: (a) the child's 
talking, listening, reading, and writing behaviors and the contexts in which those 
behaviors occur; (b) the titles of texts read b\ the child and notes about the child's 
reading progress and reaction to the particular text: and (c) information about the 
child's written work, including the context in which the writing occurred (e.g.. 
alone or in a group, with consultation from the teacher or without, type of writing, 
and the child's and teacher's response to the writing). Notes taken on this form 
are used b\ the teacher to plan future lessons and to aid in the end of the year 
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summary of the child's development on the Primary Language Record form. Use 
of the forms is flexible and does not involve applying a scoring scheme to student 
performance; rather, teachers adapt the forms as they see fit and use them 
according to their own schedule. 

Copies of the Primary Language Record Form and the Observations and Samples Form appear in 
Appendix A. 

The Primary Language Record Handbook for Teachers is a 65-page guide to help 
teachers use the record forms and learn to interpret the records they keep. : 

The Primary Learning Record 

The PLeR uses the same types of forms as the PLR except that they are designed to help 
teachers structure and assess their observations of children's learning in all subject areas. As the 
PLeR is used by teachers participating in ESAP, the forms themselves are largely irrelevant (they 
use PLR and PLeR forms interchangeably; indeed, when asked if they could provide copies of the 
PLeR forms, the teacher consultant and several Park Elementary teachers produced PLR forms 
instead): the switch from the PLR to the PLeR was motivated by teachers' decisions to expand the 
scope of their recorded observations, not so much by the forms themselves. 

F.xamples of the PLR and PLeR in Use 

The teachers at Park Elementary who have used the PLR and who are using the PLeR 
describe the stages of inquiry they go through as thev focus on a child, using the PLR to guide 
and record their observations. These stages of inquiry include: 

• How did the activity the child is involved in originate (e.g.. Did the teacher assign 
or suggest an activity? Did another child initiate the activity? Did the child 
choose the activity himself or herself?) 

• What did the child do during the activity? 

• What questions did the child raise during the activity? 

• What did the child do when he ot she got stuck? 

• Where did the child go with the activity? 

• What should the teacher do next to build on the child's learning experience with 
the activity ? 

In the Spring of IW4. two teachers at Park l.lementarv and the CCT. teacher consultant 
independent!) explained the same (or virtual!) the same) set of questions they ask themselves as 
the) observe the children the) teach 



'Published by llcmcmunn Educational Hooks Inc. Note that, to date, the Handbook and forms remain in 
their original British formats. Consequent!), some of the terminology used is unfamiliar to American readers 
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Teachers also provided examples of how they have used the PLR and PLeR to 
individualize instructional activities. For example. Park Elementary’s special education resource 
teacher talked about a boy in her class who was interested in drawing but had a very limited sight 
vocabulary. After observing his strength (drawing) and recognizing his weakness (sight 
vocabulary), she developed activities designed to draw on the strength to help improve his 
vocabulary and language skills. The teacher and student made books together. The child would 
draw a story, his teacher would then help him write the story he had drawn, and he would 
maintain new sight vocabulary words in a "word bank" at home. 

Another example was provided by the Park Elementary prekindergarten-kindergarten 
teacher. She told of how she observed a girl in her classroom playing with blocks. The child 
was talking to herself. "I'm going to put a red block here and another one here, blue here and 
here. . . ." Watching her. her teacher recognized that the girl was dealing with issues of 
symmetry and further realized what types of problems the girl was ready to deal with in math. 

The teacher also identified ways to support the child's learning through literature (e.g., repetitive 
motion, as i,i The Gingerbread Man). 

Finally, a kindergarten-first grade teacher told how she sometimes uses her observations of 
children at particular work areas in her classroom to understand not only children's learning 
better, but also to evaluate the effectiveness of how she is using those work areas. For example, 
she told of how she felt she had been ignoring the "water table" in her classroom, as it is a very 
self-sustaining activity (children play with water in a large, shallow "table” and conduct 
"experiments”). The teacher decided to sit in a corner and take notes as some children worked at 
the water table. She observed them experimenting with bubbles and recorded the children's 
"science talk" about "mixing magic potions." The children conducted an inquiry of their own 
design into why bubbles touched by wet hands didn't break but bubbles touched by dry hands did 
break. Through her observations of these children, the teacher said she better understood the 
learning that takes place at the water table and will better be able to guide children in their 
learning at that work area. 

An additional example of how one teacher uses the PLR/PLeR in her classroom appears in 
Exhibit II. 

Evaluation 



There has been no formal evaluation of the effects the PLR/Pl.eR has upon teaching and 
learning. However, the C'CE teacher consultant has completed annual reports documenting the use 
of PER during the 1992-93. 1993-94. and 1994-95 school years. The reports describe how the 
PER (and in 1994-95. the Pl.eR) was implemented in classrooms and how the author and other 
teachers have reacted to its use. She includes comments from all teachers involved at the 10 
schools participating in E.SAP. 



EXHIBIT II 



Using the PLeR to Assess Children’s Learning 



Yvonne Smith's preK-K classroom makes no waste of the modest amount of space 
available to her 20 or so young students. The classroom has about ten different work areas in 
which children build with blocks, play at the “water table." draw, paint, and write, dress up and 
play house, play at the sand table, observe the class' pet gerbils. bake, and read in the “library." 

During a two-hour period one morning in April 1995. Yvonne's students worked in 
pairs or trios at all of these activities. Teacher's aides worked with three pairs who required 
adult assistance: children baking biscuits for the class to enjoy as a snack later, children 
draw ing pictures in crayon and then ironing their drawings onto fabric, and children sew'ing 
dolls of their favorite people. Yvonne moved among the other children in the classroom, 
observing their activities and talking with them about what they were doing. 

Yvonne later talked about the observations she made of children's learning during this 
period and how she would use the PLeR to record and analyze the observations and to plan 
future activities for her students. 

In one instance, two girls playing in the playhouse called Yvonne over to sing “Happy 
Birthday" to her. After they were done. Megan, an African-American child who speaks only 
English, said. "Now we have to sing it in the other language." meaning Spanish (the children in 
the class, many of w hom speak Spanish as their first language, have been learning to sing 
Happy Birthday in Spanish). Yvonne made a note of Megan's recognition of the fact that there 
are multiple languages, and she planned to talk more about that fact the next time the children 
all sang Happy Birthday together as a group. 

In another instance. Yvonne talked with a Hispanic boy working at the sand table. She 
asked him what he had built in the sand, and he said, "a bump." Yvonne knew that this word 
was a new one for Alberto, whose English skills were still limited. She shared with him two 
other words he could use to describe what he had built, "hill" and “mountain." She then asked 
him how many words he had to describe it. He said. "Two." Yvonne held up two fingers and 
said. "Do you have this many." or. holding up three fingers, "this many?" Alberto realized he 
had said "two" when he meant "three." and Yvonne realized that he was still translating 
numbers from Spanish to English in his mind. Yvonne said that she will use what she learned 
from her observations of Alberto during this period to select reading materials that reinforce his 
growing English vocabulary ti e., books that have hills, mountains, and bumps in them) and 
that use numbers in both English and Spanish. 



I hose who use the IM.R I’l.eR stress its validity for individualized instruction. As one 
teacher put it. "Assessment is instruction." meaning that ongoing assessment of children's progress 
must guide instruction and that careful, systematic assessment designed to inform instruction, 
particularly as it is tailored to the individual child, is inherently valid, leaehers at Park 
Elementary also suggest that the PLeR is an even more valid tool for assessment than the PER. 
as. in the words of one teacher. "It feels less cumbersome, and more natural, having more pieces" 
(i.e.. observing all of a child's learning, not just extricating the language learning). 
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Because teachers use the PLR/PLeR to help them reflect about individual children, where 
they are m their learning, and how the teacher can best help the child learn, teachers — and the 

di V d I nf'S u' | thC PL * 3nd PLeR ~ are not concerned so much about reliability'. Insofar as the 
PLR/PLeR helps teachers make good choices about the instructional techniques and the curriculum 
they use with their students, the tool, they say. is reliable. Teachers also suggest that interrater 
reliability is not an issue, except insofar as teachers use it to communicate information about 
children amongst themselves and with parents. The PLR/PLeR is not used to rate a child’s 

development against others, but to evaluate each child's progress against her or his own baseline 
performance. 



Resource and Training Support 



Teachers using the PLR and the PLeR receive and give ongoing 
multiple channels: " “ 



training in its use through 



• CCE study groups, 

• CCE’s teacher consultant. 

• Continuing Education classes at Lehman College, and 

• Park Elementary staff meetings. 

, hc basic struclure of support activ ities has not changed since the introduction of the PLeR. 
CCE Study Groups 



The CCE coordinates tea, her "study groups." which take two forms. One studv group, 
which meets every other week, involves the Park Elementary teachers who use the PLR/PLeR 
coming together over lunch to discuss their use of the forms and the impact the tool has on their 
teaching. I hey share ideas that facilitate their use of the PLR/PLeR. for example, one teacher 
found it cumbersome to maintain PLR/PLeR records on all 30 children in her class, because the 
forms were rarely handy when she wanted to record something. She started keeping pads of 
white stickers with her. When she wants to record an observation, she writes her notes on one of 
the stickers. At the end of the day. she transfers the stickers onto the observation forms. 



More specifically, however, these groups meet to share problems or points of interest thev 
have identified in their classrooms, generally as a result of using the PLR PLeR. I or example, a 



though it is not directly relevant to the current case studv. a related initiative in California the 
tali ornia Learning Record, has adapted the PI R PLeR tor use by California educators The croup Support, nu 
tcachcrs participating in the work there has performed an analysis of interrater reliabilitv w ith respect to teachers' 
assessment, of children s reading levels The group found very high interrater reliabilitv. However this group 

r ~ r m T M V7 bl ° u ,C> NVCrC looking al ,hc lcvcl al which children were reading as designated 

h ? , ~ CrS par,,apa,m S in NYAN d » "«>« attempt to identify children's reading (or 

other) development according to any particular scale. Thus, educators and policvmaken who are interested in 
uniformity of application of PLR PLeR and mterra.er reliability of teacher judgments should refer to the work of 
he C a ,1,-mta group. Barr. M A & Sy verson. M. (1994). The ( ahfonua Learn,,,, Reeoni System Moderate 
Ri.adinf’s Report. San Diego: University of California. 
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teacher mmht share an insight into one of her students, and the group may discuss possible ways 
of best instructing that student. Frequently, teachers say. by discussing the educational needs of 
one student, they acquire insights into many others. 

In addition to the Park Elementary study groups, approximately once a month CCE brings 
together teachers from different member schools to share experiences. CCE. Park Elementary, 
and the developers of the PLR/PLeR all stress that communication among teachers is crucial to 
improving the craft of teaching. The CCE consultant who works with teachers in study groups 
calls the sessions "a shot in the arm for teachers." One teachei commented about the 
collaborative nature of the endeavor: "As teachers begin to do this [talk about their students 
learning), they become each other’s gurus. 

During the 1994-95. two Park Elementary teachers, together with the CCE teacher 
consultant, led a studv group for CCE teachers using the PLeR. According to the CCE teacher 
consultant, the structure of the study group they led generally included a discussion of an assigned 
article, reviews of evidence collected by teachers using the PLeR. and a discussion of other issues 
identified by teachers. 

Th«> rrF. Teacher Consultant 

The CCE teacher consultant who coordinates the teacher study groups olten works one-on- 
one with teachers, visitinu their classrooms and sharing her observations. Frequently teachers 
invite her to observe arid comment on a single child they are having a hard time getting to know. 

The teacher consultant says that she works one-on-one with teachers primarily at schools 
othc than Park Elemcntarv: teachers less accustomed to observing children closely need her 
assistance more. The consultant works with a child to learn something about the child’s leamints 
Mvle. She then meets with the teacher or group of teachers to discuss her observations. Soon, me 
teachers are comfortable making the observations themselves: "As you develop a collection ot 
these profiles, the teachers start doing it themselves .... I go in to raise teachers’ questioning. 

Continuing Education Classes for PLR Teachers 

Thtough one of the other member groups of NYAN. the Elementary Teachers Network, 
teachers usinu the PLR/PLeR tin conjunction with any of the three groups) may enroll in 
continuing education classes (e g., one Park Elemcntarv teacher took a course on the "Reflective 
Practitioner" during the spring of 1994) at Lehman College. These classes ty pically meet once 
every month and are free to teachers working with NYAN member organizations. 

Park Elementary s Ow n Staff Development Activities 

I very other week on their own time. Park Elementary teachers have a statl meeti g. I he 
meet ini; is not business-oriented (those meetings occur at other times). Rather, teachers come 
together to talk about the children in their classrooms. I his tradition is a longstanding one at 
Park Elementary, and it illustrates why and how the PLR came so naturally to Park Elementary s 
teachers. These meetings resemble the study groups., ibe only difference being that they are not 
focused specifically on the use of the PI R Pl eR 
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Impact of the PLR and PLeR 



Because the PLR and PLeR are consistent with the educational philosophy already 
espoused by Park Elementary teachers, their impact on the school's teachers and students, as 
distinct educational tool ;, remains difficult to evaluate. To those who use it. the PLR/PLeR has 
helped teachers focus their observations of children and maintain useful records of their 
observations; the PLR and PLeR have provided a structured approach for what teachers were 
doing already. 

Impact on Teachers 

The teachers who have used the PLR and who are using the PLeR have all volunteered to 
do so. recognizing in the technique something they wanted to do in their own classrooms. They 
saw the PLR as a promising tool to supplement what they were already doing, and the PLeR 
proved an even more natural fit. Teachers at the school who do not use the PLR or PLeR are not 
opposed to their use; they merely did not identify in it enough value to add it to what they were 
already doing, As one teacher said, "We were doing PLR before we had PLR.'* 

I sing the PLR/PLeR requires a significant amount of time and discipline on the part of 
teachers. One teacher said she reviews and organizes the notes she has taisen during the day while- 
traveling home on the subway. This teacher also likes the fact that the PLR and PLeR are what 
she calls "non-form forms.” meaning that she can adapt he system to her own teaching style. 
Another teacher echoes this sentiment, saying. "They encourage you to make any changes you 
want to make. That's one of the things I like about it. I play with it. If it isn't useful to me. 
why use it?" 

This teacher also say s that, though she does not use the PLeR with all her students, she 
does use it with several: for these students, she keeps a notebook w ith several pages devoted to 
each child. When she observes something she wants to record, she turns to the child's section in 
the notebook. Still another teacher says she makes sure she observes each child in her classroom 
at least one time each week. These routines help the teachers ensure that they are making good 
use of the PLR/PLeR. 

Teachers differ with respect to how much they believe using the PLR and PLeR has 
affected the way in which they work with children. One teacher said that using the PLR allowed 
her to "Tine tune" her teaching Another, however, said that using the PLR bolstered her 
confidence in her ability to observe accurately what her students are doing and learning: "I now 
trust what I see and what I hear." This teacher also values collaborative teaching more than she 
did formerly, stating. "I know that it's important to share |my observations] with colleagues, 
parents, and children." Both of these teachers in Spring I SOS said that using the PLR — ■ and now 
the PLeR - gets easier each year as they become more used to and better at the process. 

inally. teachers also observe that the record keeping they do with the PLR I’l.eR assists 
them n writing the narrative reports that they complete for each child two times a year. 

Teacher Response to Subst itu ting the PLeR for the PLR . Teachers at Park Llementary 
say that they took the switch to the PI el< in stride. Indeed, just as some Park Elementary 
teaehets said that they were "doing PIR' before there was a PLR. teachers who used the PLR 



also said that they had already begun to use it to track children's progress in all subject areas 
before they actually started working with the PLeR. Thus, the switch to the PLeR came naturally 
to them. 

Comments from the ( CE Teacher Consultant . The CCE teacher consultant who works 
with teachers using the PLR/PLeR. asserts that the experiences of Park Elementary teachers are 
atypical. Teachers at Park Elementary come to the PLR/PLeR far better equipped to use it than 
do most of the teachers she works with at other schools. Park Elementary teachers have several 
factors working in their favor. The school has always valued teachers coming together to talk 
about children and to figure out ways to meet their needs; consequently, the school has a flexible 
schedule to allow teachers time to meet, and teachers are willing to put in significant amounts of 
their own time to work with their colleagues. 

Teachers at other schools have, according to the CCE teacher consultant, struggled with 
the PLR. Whereas the vision of education held by Park Elementary teachers led some of them to 
choose to use the PLR. and then the PLeR. at some schools it is using the PLR that has led 
teachers to struggle to alter their vision of educational practices. Says the teacher consultant, 

"The PLR works in places w ith administrative support and a vision of the way ch.ldren and 
teachers learn. It does not work where there's not a beginning of that vision." She also asserts. 
"Time and space have to be legislated for teachers to talk to each other." accommodations already 
in place at Park Elementary when the PLR was first introduced. According to the teacher 
consultant, some teachers at other schools have dropped out of ESAP because “they can't be seen 
as good teachers in their schools and look at children the way we do." 

The CCE teacher consultant and a Park Elementary teacher who served as a mentor to 
other teachers using the PLR during the 1993-94 school year spoke to how thoroughly many 
teachers have embraced the PLR/PLeR. Typical remarks they hear are. "I never really listened to 
children before. 1 never really watched them work. I never really paid attention to their choices." 
The mentor teacher says that pme of the teachers she talks to have had to "dramatically change 
their classrooms" and that the; are . . now furious with Boards of Education and the institutions 
that trained them — the beginning point is the child/' 

Impact on Curriculum and Instruction 

As the PLR. and then the PLeR. joined an educational program already very child- 
centered. their impact on curriculum and instruction at Park Elementary has been mostly at the 
margin. One teacher says. "Because I have written down what the child is doing. I know what 
can possibly come next. I know what kinds of materials 1 need. The PLR helps me individualize 
instruction more. . . . But there is no huge change 1 in my instructional methods], only fine 
tuning." Other teachers at the school also commented that the PLR/PLeR has helped them 
individualize instruction for their students and identify which educational materials students need. 
Also, during 1993-94. one teacher of fifth and sixth graders at Park Elementary used the PLeR 
(prior to its adoption by all ESAP-participating teachers) to help her restructure her mathematics 
curriculum, integrating it with other subject areas. 

Teachers' comments about the impact of the PER/Pl.cR in this respect did not change 
from 1993-94 to 1994-95, 



11-12 



55 






Impact on Students 



According to teachers, students take the PLR/PLeR in stride. One teacher reported that 
her children often ask her. “'Are you going to write down what I did?" All teachers reported how 
using the PLR/PLeR has helped them tailor their lessons to their students' needs; some of their 
anecdotes have already been included, above. 

Impact on Students with Special Needs . One significant use teachers at Park 
Elementary put the PLR to in 1993-94 was to aid them in their documentation while in the 
process of declassifying special education students. Park Elementary's special education teacher 
reported that she used the PLR as part of her documentation of three students' readiness to be 
declassified as special education students; two of these students were declassified during the 
school year. This teacher also believes that the PLR/PLeR would make a good substitute for 
lEPs. which she feels are too rigid and cumbersome: the PLR/PLeR could serve the same purpose 
but allow more flexibility. 

Impact on Parents 

Teachers report that parents like the PLR/PLeR. and that they especially like the parent- 
teacher-student conferences devoted to discussing the child's literacy and other intellectual 
development; several parents have told teachers they've never been asked before to share their 
know ledge of their children. Teachers and parents alike are gratified to discover that their 
observations of their children are so similar. (Note, however, that these parental reactions come 
filtered through teachers; .he one parent we interviewed said that her daughter's teacher does not 
use the PLR or PLeR. It is. therefore, difficult to determine the impact the tools have had on 
parents.) 

Neither teachers nor the parent interviewed for this study commented on any changes in 
parental reactions between 1993-94 and 1994-95. 



Future Plans 



Now that the PLeR has replaced the PLR. teachers at Park Elementary who use the tool 
no longer have plans to change their use of the technique. They w ill continue to use the PLeR as 
it makes sense to them in their classrooms. As a school. Park Elementary has no plans to expand 
use of the Pl.cR. Any teacher who wants to use it is welcome to do so. and no teacher is 
compelled to try it. 



Conclusions 



Park Elementary is a school in which teacher inquiry and teacher collaboration are 
encouraged. In such an environment, the interweaving of assessment with curriculum and 
instruction — as fostered by the PLR. the PLeR. or other techniques — comes naturally to 
teachers, at least after practice. 



The PLR/PLeR is an example of an assessment technique designed, more than anything 
else, to inform curriculum and instruction. It encourages teachers to hone their observational 
skills and furthers their engagement in the pedagogical process. It is a tool for teachers to apply 
in their classrooms to help them work more closely with their students. It is not an assessment 
technique that aims to provide information about students' progress toward any pre-specifiou 
standards of achievement or performance, although, based on their observations, teachers seem to 
formulate individualized goals for their students. Given various attributes and limitations, it is 
hard to imagine the PLR or the PLeR being used successfully for purposes of accountability or 
large-scale assessment of student achievement. Simply put. when teachers commit their time to 
using the PLR or PLeR to guide their classroom activities, the process most likely is, as the CCF 
teacher consultant suggests, a "powerful” tool and far "more profound than it sounds." 
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APPENDIX A 



Primary Language Record Form 



Primary Language Record 



Di 

O 

ERIC 



School 




School Year 


Name 


DoB 


Summer born child Q 




□ Boy □ Girl 




Languages understood 


Languages read 




Languages spoken 


Languages written 




Details of any aspects of hearing, vision or coordination 
affecting the child's language/literacy. Give the source 
and date of this information. 


Names of staff involved with child's language and literacy 
development. 


IP 3 l*t To be completed during the Autumn Term 


A1 Record of discussion between child's parent(s) and class teacher < Handbook pages 12-13) 




Sianed Parentis) 


Teacher 




Date 


A2 Record of language/literacy conference with child < Handbook pages 14-15) 




Date 
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^nlr* tor Language in Primary Education, ISM. Published m the U.S.A. by Halnamann Educational Books, Inc., 361 Hanover 8t., Porlamouth, NH 03001 



«_ n To be completed during the Spring Term and to include information from all teachers currently teaching 

(*3li D the child. 

Child as a language user (one or more languages) ” (Handbook p»geti7-w 

Teachers should bear in mind the Authority's Equal Opportunities Policies (race, gender and class) in competing each section of 

%uZon7op£«unl,£, lor AIK, th. ILEA ..port on spoci.1 «.uca«on,l needs. 

(Handbook pages 19-22 ) 

experience and confidence in talking and listening with different people in different settings. 



What experiences and teaching have helped/would help development in this area? Record outcomes of any discussion with 
head teacher, other staff, or parent(s). 



B2 Reading (Handbook pages 23-28 

Please comment on the child's progress and development as a reader in English and/or other community languages: the stage 
at which the child is operating (refer to the reading scales on pages 26-27); the range, quantity ^and variety of fading | 
areas of the curriculum; the child's pleasure and involvement in story and reading, alone or with others; the range of strategies 
used when reading and the child's ability to reflect critically on what is read 



O 
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continued 






B2 (continued) 



What experiences and teaching have helped/would help development in this area? Record outcomes of any discussion with 
head teacher, other staff, or parent(s). 



B3 Writing (Handbook pages 29-34) 

Please comment on the child's progress and development as a writer in English and/or other community languages: the degree 
of confidence and independence as a writer; the range, quantity and variety of writing in all areas of the curriculum; the child's 
pleasure and involvement in writing both narrative and non-narrative, alone and in collaboration with others; the influence of 
reading on the child's writing; growing understanoing of written language, its conventions and spelling. 



/ 



What experiences and teaching have helped/would help development in this area? Record outcomes of any discussion with 
head teacher, other staff, or parent(s). 



Signature of head teacher and all teachers contributing to this 
section of the record: 



BT 



( Handbook page 35) 



PartC To be completed during the Summer Term* 
Cl Comments on the record by child's psrent(s) 



C2 Record of langusge/literacy conference with child 



SaSSSSttS. .eache, is as up ,o date as po*b,e. Pla.se comm,™ on chants 

and development in any aspect of the child's language since Part B was completed. 



What experiences and teaching have helped/would help development? Record outcomes of any discussion with head teacher, othe 
staff, or parent(s). 



Signed : Parent(s) Class Teacher - 

Q ate Head T eacher - 

J •Jobe completed by the Summer half-term for 4th year juniors. 

s± 



| Observations and Samples (Primary Language Record) 



attach extra pages where needed 



Name: 



Year Group: 



1 Talking & listening: diary of observations 

The diary below is for recording examples of the child's 
developing use of talk for learning and for interacting with 
others in English anchor other community languages. 

Include different kinds of talk (e.g. planning an event, 
solving a problem, expressing a point of view or feelings, 
reporting on the results of an investigation, telling a 
story...) 

Note the child's experience and confidence in handling 
social dimensions of talk (e.g. initiating a discussion, 
listening to another contribution, qualifying former ideas, 
encouraging others . . .) 

The matrix sets out some possible contexts for observing 
talk and listening. Observations made in the diary can be 
plotted on the matrix to record the range of social and 
curriculum contexts sampled. 

(Handbook pages 37-39) 





SOCIAL CONTEXTS 


LEARNING 

CONTEXTS 


pair 


small 

group 


child 

with 

adult 


small/large 
group 
with adult 




collaborative reading 
and writing activities 












play, dramatic play, 
drama fit storying 












environmental 

studies 

& historical research 












maths & science 
investigations 












design, construction, 
craft & art projects 

















































Dates 




Observations and their contexts 
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for Language in Primary Education, 1981 Published in the U.8.A. by Hetnemenn Educational book a, Inc., 361 Hanovar tt., Portsmouth, MM 03801 






2 Reading and Writing 

(reading and writing in 



: diary of observations 

English and/or other community languages) 



{ Handbook pages 40-44) 



Rea ding 

Record observations of the child's development as a reader (including wider experiences of story) across a 
range of contexts. 



Writing 



Record observations of the child's development as a writer (including stories dictated by the child) across a 
range of contexts. 



■ 3 Reading Sample* (reading in English andtor other community languages) 

£ to include reading aloud and reading silently (Handbook pages 45-49 f 


■ Dates 




|P Title or book/text (fiction or 

information) 

1 




Known/unknown taxt 

| 




P Sampling procedure used: 

f- informal assessment/running 

| record/m iscue analysis 


| 


• 




Overall impression of the 

■ child's reading: 

■ e confidence and degree of 

. independence 

• involvement in the book/ 

■ text 

e the way in which the child 
read the text aloud 








Jj Strategies the child used 

when reading aloud: 
r _ • drawing on previous 

1 experience to make sense 

■ of the book/text 

* playing at reading 

_ * using book language 

■ * reading the pictures 

■ * focusing on print 

(directionality, 

'm 1:1 correspondence, 

■ recognition of certain words) 

■ e using semantic/syntactic/ 

grapho-phonic cues 

■ e predicting 

1 • self-correcting 

• using several strategies or 
over-dependent on one 




<4 




^ Child's response to the 

book/text: 

■ • personal response 

■ • critical response 

(understanding, 
evaluating, appreciating 
| wider meanings) 

| 








B What this sample shows 

about the child's 

s ■ development as a reader. 

* ■ Experiences/support 

needed to further 
| development 
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ERIC • Early indicators that the child is moving into reading 



Observations and Samples (Primary Language Record) 



attach extra pages where needed 



Name: 



Year Group: 



1 Talking & listening: diary of observations 

The diary below is for recording examples of the child's 
developing use of talk for learning and for interacting with 
others in English and/or other community languages. 

Include different kinds of talk (e.g. planning an event 
solving a problem, expressing a point of view or feelings, 
reporting on the results of an investigation, telling a 
story...} 

Note the child's experience and confidence in handling 
social dimensions of talk (e.g. initiating a discussion, 
listening to another contribution, qualifying former ideas, 
encouraging others . . .) 

The matrix sets out some possible contexts for observing 
talk and listening. Observations made in the diary can be 
plotted on the matrix to record the range of social and 
curriculum contexts sampled. 

(Handbook pages 37-39) 
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LANGUAGE ARTS AND MATH PORTFOLIOS: 
NINOS BONITOS ELEMENTARY SCHOOL 



Introduction 

This case study examines a performance assessment system designed at an individual 
school in response to its own pressures for reform. The subject of the study, Ninos Bonitos 
Elementary School, is located in suburban Ninos Bonitos, California, and is part of the San Diego 
Unified School District (SDUSD). As of the 1994-95 school year, Ninos Bonitos Elementary 
School served a total population of 924 students from pre-kindergarten through sixth grade. 
Thirty-nine percent of the student population are of Southeast Asian heritage (Vietnamese, 

Laotian, and Hmong), 46 percent are Hispanic, and 5 percent are African American. Other ethnic 
groups include East Asians and Filipinos. Almost 95 percent of the student population qualify for 
free or reduced lunches, and 77 percent have been identified as possessing limited English 
proficiency (LEP). Eighty-six percent qualify for Chapter 1 services (statewide, only 18 percent 
of the student population are eligible for Chapter 1 services). The five major languages spoken 
by students are Vietnamese, Hmong, Lao, Spanish, and English. 

Ninos Bonitos is located in a suburban community close to the University of San Diego. 
The neighborhood immediately surrounding the school served as a relocation site for Lao and 
Hmong refugees arriving at nearby Camp Pendleton in the mid-1970s. It is an area that still 
st.nggles with poverty and its related problems. As an indication of the poverty faced by many of 
Ninos Bonitos' children and families, the school is seriously considering purchasing uniforms for 
all students to wear, since so many families cannot afford to clothe their children adequately. 

Although the study focuses on reforms undertaken at the local level, it first includes some 
discussion of district and state reforms that interact with and have an impact upon local initiatives. 

Participants 

At Ninos Bonitos, the people identified in Exhibit I were interviewed. 

Observations 



In addition to conducting interviews with the listed participants, during the 1993-94 school 
year the study staff observed a four-hour staff development session that included both a 
presentation by the mathematics curriculum committee responsible for developing the school's 
mathematics performance assessment system and a video explaining California's impending state- 
wide performance assessment system. In 1994-95, the study staff observed a third-fourth grade 
(middle wing), transitional classroom for four hours. 



EXHIBIT I 



Study Participants 




District assessment specialist 
School principal’ 

School vice principal 
A school resource specialist 
The school's teacher union 
representative’ 

One mathematics teacher (early 
childhood wing) 

One parent who also serves as the 
school’s parent volunteer coordinator 



District assessment specialist 
One school board member 
School principal’ 

A second school resource specialist 
The school's teacher union 
representative’ 

Two language arts teachers (early 
childhood and upper wings) 

One mathematics teacher (upper 
wing) 

One parent 



District and State Context 



In 1994-95, San Diego Unified School District included 109 elementary schools, 21 
middle schools, 16 hi^h schools, and 7 “atypical” schools, all together serving approximately 
130,000 students. Approximately 17 percent of the SDUSD student body is African American, 18 
percent is Asian, 31 percent is Caucasian, and 33 percent is Hispanic. Close to 60 percent of the 
students qualify for Chapter 1 funds, and about 70 percent receive free or reduced lunch. 

Under both its former and its new superintendent, the SDUSD has been a major advocate 
of innovation and reform, and it has taken a particular interest in developing and implementing 
alternative assessments of student performance. In 1991, Assembly Member Quackenbush 
introduced AB 40, a local piece of legislation designed to fund research, development, and 
dissemination of alternative assessments. The California Assessment Collaborative (CAC), 
composed of schools, districts, and county offices in northern California and the San Diego 
Consortium (consisting of San Diego City and San Diego County schools), was invested with the 
responsibility for carrying out these three functions. 

Under the San Diego Consortium, the SDUSD became involved in several assessment 
projects, with teachers taking the lead in developing the actual assessment tools. Those projects 
include: 



Chapter 1 Portfolio Assessment Development Across Levels; 



Dance Portfolio Assessment Project; 



This position changed hands between the first and second years of study. 
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* K-4 Learning Assessment Project; 

• Performance Assessment Coliaboratives in Education (PACE) Portfolio Project; 

* Standards and Assessments for English Learners; and 

• The University of California at Los Angeles Center for Research in Educational 
Standards and Student Testing (CRESST) U.S. History Project. 

According to the district assessment specialist, although all of these projects generated a 
great deal of enthusiasm and interest district educators questioned the wisdom of participating in 
numerous, isolated projects, rather than building a unified district capacity to confront the task at 
hand. Hence, the district became linked with several major national movements to promote 
performance assessments, most notably the New Standards Project (NSP) 2 and the National 
Alliance for Restructuring Education, part of the New American Schools Development 
Corporation (NASDEC). 3 Although these affiliations do not provide the district with large 
financial resources, they do provide access to much-needed expertise and models for educational 
reform, particularly in the areas of curriculum, instruction, and assessment. 

At approximately the same time as SDUSD became allied with these national partners, its 
own district-sponsored Accountability Committee of educators, parents, and citizens launched a 
“homegrown” network of 10 (and later 25) schools, which volunteered to serve as models in a 
Leadership in Accountability Demonstration (LAD) project. The project focused on increasing 
school accountability through the mechanisms of improved standards, new forms of assessment, 
and improved public reporting. Although all of San Diego's 160 schools are eligible to benefit 
from National Alliance and NSP expertise, these 25 LAD schools are the leaders in testing and 
implementing National Alliance and NSP reforms and will serve as mentors to other district 
schools through a cluster feeder pattern. 

The National Alliance has promoted district adoption of five “design tasks” for improving 
student achievement and organizational effectiveness. The first design task deals with curriculum, 
instruction, assessment, and technology. SDUSD's new superintendent has structured her district- 
wide reform efforts around these five design tasks, and over the past year she developed 16 
expectations or outcome measurements to test, objectively, whether or not progress is being made 
in each of the five areas. 

Unfortunately, on of the most critical instruments for measuring student achievement — 
results on the new California Learning Assessment System (CLAS) state test — is no longer 
available. The CLAS was initiated by the state legislature in 1991 as a new authentic assessment 
of higher-order thinking in the major academic subjects and was piloted in California schools in 
language arts and mathematics in 1992-93 and 1993-94. The state test was to consist of open- 
ended and enhanced multiple choice problems that students would work on in groups and then 
individually respond to in writing. Students would not only have to solve the problems, but 
would have to explain or justify their solutions as well. However, California's governor vetoed 



‘California is one of 19 NSP state partners, and c an Diego is one of six urban school district partners. 
’San Diego is one of 9 National Alliance sites, all of which are also NSP partners. 
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the legislation last summer as a result of pressure from conservative groups and special education 
groups who were concerned about the assessment's content, method of administration, and equity. 

The district's assessment specialist, as well as the school board member interviewed, noted 
that the district has been left somewhat “rudderless” by the demise of the CLAS, since CLAS 
results were to be the basis for measuring progress in meeting the National Alliance design tasks 
and for measuring success among the LAD schools. The school board member said that school 
officials felt quite strongly that the CLAS would be a better measure of school improvement than 
what has traditionally been used — the standardized Abbreviated Stanford Achievement Test 
(ASAT.) The superintendent is still a strong proponent of the use of performance assessments 
within the district; within a week of the CLAS veto she had issued a report to the school board 
regarding the status of performance assessment in the district, advocating increased use of NSP 
assessments by all schools in the district as a replacement for the CLAS. In the spring of 1995, 
district staff conferred with NSP personnel about using the NSP standards and assessment 
strategies in every school in the district, thus making SDUSD as a special “action research 
laboratory” for the NSP program. 



School Context 

Considered a leader in assessment reform, Ninos Bonitos Elementary School is a LAD 
school, and its Principal (the Principal during the round one visit) was at one time on the LAD 
committee. In addition, Ninos Bonitos is one of eight “1274” schools in the district schools that 
are part of a legislatively mandated state network; network comprised of schools that are leaders 
in reform and restructuring efforts. Through its affiliation with LAD and the 1274 program, 

Ninos Bonitos draws heavily upon the National Alliance and NSP approaches and is continually 
asked to share its assessment practices with the rest of the district. Because Ninos Bonitos' 
history of reform places it in the vanguard of SDUSD schools, the school is, at this point, giving 
more to the district than it is getting in the way of technical assistance and expertise. 

The . nool occupies two separate sites and operates on a year-round schedule, employing 
two adminisuHtors, 30 classroom teachers, and 52 instructional aides. In addition, the school 
maintains a full complement of support staff, including resource teachers, a librarian, a counselor, 
a nurse, a volunteer coordinator, community aides, and custodial staff. Although nearly half of 
the staff speak a second and often a third language, only two credentialed classroom teachers 
speak any Southeast Asian languages (one speaks Vietnamese and the other speaks Hmong). 

Ninos Bonitos’ elementary school is democratically governed through a staff committee 
structure coordinated by the school principal. All certified and classified staff must serve on both 
a site committee (e.g., student involvement, grants, finance, personnel) and a curriculum 
committee. Committees make all major decisions related to the school's operations, although 
committee recommendations are always submitted to the full staff for a consensus review. 

Ninos Bonitos' Principal in i993-94 was described by the teachers included in the study as 
an “excellent leader.” She initiated the school's comprehensive restructuring process in 1988, and 
recently was awarded the honor of “California Principal of the Year.” At the end of the first year 
of this study, she was chosen in a national competition for the position of Principal in Residence 
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with the U.S. Department of Education. As a result, the school acquired a new Principal in 1994- 
95, the second year of the siidy. 

History of School Reform 

The development of the school's performance assessment has been heavily influenced by 
the school's demographic character. In 1988, after six months of careful observation, the school 
principal determined that children's needs — particularly for learning the English language — 
were not being met appropriately. Ninos Bonitos' ASAT scores 4 were extremely low, and staff 
morale was qv te poor She felt more appropriate instructional programs and language proficiency 
groupings were needed to improve learning and to provide greater equity of services to Ninos 
Bonitos diverse student population. In addition, as an early proponent of site-based management 
and accountability, she felt the full staff should take responsibility for developing the mechanisms 
of change. 



To initiate the restructuring process, the Principal developed a site-based committee 
structure of governance for the school, which proceeded to revamp the school curriculum and to 
reorganize the school day. Emphasis was placed initially upon language arts, historically the area 
of students' greatest weakness and of staff's greatest frustration. Students were organized into four 
nongraded but age-appropriate “wings.” At the school's Annex, pre-kindergarten, kindergarten, 
and first graders who are severely handicapped or in need of developmental services would 
participate in a special early childhood education program. The primary (grades i and 2), middle 
(grades 3 and 4), and upper (grades 5 and 6) wings were housed at Ninos Bonitos' main campus. 

Ninos Benitos new organizational system worked very well and was adopted permanently 
in 1989. In the morning, all students in each wing (except those who are Spanish-speaking) are 
assigned to one of six levels of English proficiency (e.g., entry, Sheltered A, Sheltered B, 
Transition A, Transition B, and non-sheltered) for language arts and mathematics coursework. 

For example, a first grade student with limited English proficiency might be assigned to a primary 
wing Sheltered A classroom, while a 5th grade student with near fluency in English might be 
assigned to an upper wing Transition B classroom. Spanish-speaking children are enrolled in a 
bilingual program, in which all three basic subjects are taught in Spanish, supplemented by 
additional English-language development. In the afternoon, a reading recovery course is offered 
to first graders who are experiencing difficulty with beginning reading. 

In the early stages of her restructuring efforts at Ninos Bonitos, the school's principal 
made sure to discuss her plans with district personnel and to obtain their “buy-in.” As a result, 
her efforts encountered almost no resistance from the district and, in fact, received a great deal of 
focused attention and support. Her job was made easier by the fact that the school's 
superintendent and staff were themselves struggling to conceptualize and implement a number of 
important educational reforms. 



4 Euch year, as a "Chapter 1 school," Ninos Bonitos administers this test to all English-speaking students in 
reading comprehension and mathematics. Spanish-speaking students take r Spanish equivalent of the test, called 
the "Aprcnda". "Sheltered" students speaking primary languages other than English take the ASAT only after 
attending Ninos Bonitos for one year. 
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The school's new principal in 1994-95 is extremely impressed by what he has “inherited” 
at Ninos Bonitos. He believes the key to the school's success is empowering the whole faculty to 
serve as instructional leaders and to experiment with “cutting edge” instructional and assessment 
reforms. 

Performance Assessment Development and Characteristics 

Curriculum and assessment reform at Ninos Bonitos also began with the language arts 
program. Staff felt it was important to establish clear entry and exit criteria for every child at 
each age and language development level as well as clear instructional goals for each level. They 
wanted to be able to continuously monitor the progress of each child so that an LEP review team 
could move children to new language development levels at any time during the school year, as 
those children became ready to advance. 



To meet their objectives, faculty “wings” began to meet in 1988 to brainstorm the 
development of language arts learning outcomes and scoring rubrics — a process that ultimately 
took four years to complete. Learning outcomes, instructional materials, and portfolio contents 
were developed for each language development level within each wing (e.g., Primary Level/Entry, 
Primary Level/Sheltered A). For each level, the learning outcomes describe specifically what 
students should know and what they should be able to do with regard to oral language, reading, 
and writing in order to complete a particular level. 

If teachers keep these learning outcomes in mind, assessment becomes an integral, daily 
part of instruction. For example, an entry level LEP student at the primary level should be able, 
among other things, to use proper words to identify objects through oral language. In reading, 
that same student should be able to answer simple questions regarding the structure and meaning 
of a story. 

With the help of experts such as Dennie Wolf and Grant Wiggins, Ninos Bonitos teachers 
agreed that portfolios of student writing, along with observations on students' reading and spoken 
language collected throughout the year would be another exemplary way to measure growth in 
language arts. Teachers then spelled out what each portfolio should contain for each age and 
language development level. For instance, the primary level portfolios for Spanish-speaking 
students should contain three daily journal samples, a minimum of three independent writing 
samples, and a reading and oral language usage checklists. 5 (The portfolios then are scanned to 
generate electronic portfolios). 

In the second year of our the study (Round 2), teachers begrn to reevaluate the way in 
which the language arts electronic portfolios are constructed. To date, teachers have selected the 
student pieces to be included in each portfolio. However, they are beginning to feel that students 
should have a greater hand in the selection process, so that the portfolio becomes a tool for 
student self-reflection, as well as a method of measuring student progress from the teacher's 
perspective. This year, for instance, the middle wing teacher observed in the study allowed her 



'Reading and oral language checklists were developed as an observational tool, in order to measure student 
growth in these areas three times a year. 
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students to choose their “writing process piece” for their electronic portfolios after the first 
grading penod. The observation is described below in Exhibit II. 



Scoring rubrics were developed by teachers as a way to compare and communicate 
students academic progress without having to “label” children with standard letter grades, 
^guage arts coring rubrics were modified and refined between the two years of the study; by 
1994-95, the faculty had identified five performance levels for oral language and reading, and six 
eve s for written language. These rubrics cut across language development and age levels. In 
writing, for instance, a student can be evaluated as a pre-writer, emergent, developing, beginning 
experienced, or exceptionally experienced writer. Specific (and different) criteria were determined 
by age and performance level. Appendix A provides a copy of Ninos Bonitos'language arts 
scoring rubnc The rubrics themselves were used to redesign the student in language arts, which 
was renamed the student growth record” (see Appendix B). Students are now assigned a rubric 



EXHIBIT II 



Classroom Observation 



In the 3rd/4th-grade (“middle wing”) classroom observed, “transitional” students (those 
who are almost, but not quite, fluent in the English language), spent most of the morning working 
in cooperative groups of four, using six personal and laptop classroom computers. Their teacher 
had received five days of special training in software technology, as one of the school's five 
Apple Classroom of Tomorrow” (ACOT) teachers. As a result of her training, she had designed 
“P erf °™ an c ce task that rec f u,red students to describe and illustrate a book they had read about the 
1 ficulties Southeast Asian students (like themselves) experience as they assimilate into their new 
American culture. She adopted the role of “coach,” as specified in her training, and circulated 

throughout the classroom as her students worked - helping them with their writing as well as 
with their software skills. 6 

i a 6 ™t Sed 81631 en,husiasm for ,he “student-centered” approach she had 

earned through the ACOT program and she had developed a number of creative ways to integrate 
echnology into her language arts teaching and assessment methods (for example, earlier in the 
>ear she had asked students to write haiku poems to appear on T-shirts they designed on their 
computer screens)^ Students saved their work in both electronic and “hard-copy” versions of their 

anguage arts portfolios, which were shared with parents at parent-teacher conferences, which 
occur three times a year. 



score by their language arts teacher based on both a review of a written language portfolio and an 
assessment of performance on reading and oral language. b P 

Teachers interviewed noted that the language arts rubrics will most likely undergo 
continued refinement as the faculty's experience with them and exposure to other models 
increases. In the second year of the study, for instance, teachers were discussing refinements that 
would make tneir rubrics more consistent with the NSP standards. 
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Mathematics 



Language arts standards and rubrics have been in use for four academic years, and their 
electronic versions for three. In 1993-94, Ninos Bonitos staff began to research and write 
mathematics standards and rubrics, using the same process that they had used for language arts. 

The newly developed California state frameworks and curriculum guidelines and the National 
Council of Teachers of Mathematics (NCTM) standards were used as guides to identify eight 
critical content areas of the mathematics curriculum that students need to master, including: 
number sense and numeration, whole number operations, whole number computation, geometry 
and spatial sense, measurement, statistics and probability, patterns and relations, and fractions and 
decimals. Using NCTM materials, staff also identified five process skills necessary to understand 
the subject (e.g., problem solving, communications, reasoning, connections, and estimation). 

Students in each wing complete 22-day mathematics instructional rotations organized 
around the NCTM content areas. For instance, the middle wing transitional class observed in 
1994.95 begins the year with a division and multiplication rotation taught by their own homeroom 
teacher, and then, after 22 days, the class rotates to a measurements unit taught by a second 
teacher, and so on, so that they complete seven content area rotations by the end of the year. 

Each teacher, then, teaches his or her “specialty” rotation seven times over the course of riu^ear 
to seven different groups of students. 

From the process and content skills they developed, the mathematics committee generated 
a scoring matrix that was used in place of the current mathematics portion of the growth record to 
evaluate student progress for the first time in 1994-95. Appendix C presents a copy of this 
matrix. Students are identified as either early emergent, emergent, developing, moderately 
experienced, experienced, or exceptional mathematicians within the seven critical content areas 

identified above. 

For the early childhood wing, the committee also developed a list of activities and projects 
for each level — the idea being that teachers could use these to assess student mastery of each 
process skill within each content area. For example, to demonstrate problem-solving skills for the 
patterning content area, kindergartners must show how they can continue the formation of a 
pattern of paper apples the teacher has begun. To demonstrate pattern reasoning, they must be 
able to reply verbally to the question, “What comes next”? 

After considering mathematics rubrics, the entire faculty, meeting in groups by wings, 
went on to develop learner outcomes for each age level — outcomes that conceptualized what 
students should know in order to demonstrate mastery of each content and process skill area. The 
mathematics curriculum committee then developed separate observable student behaviors to 
accompany each set of learner outcomes. These observable behaviors specifically detail tasks 
each student must be able to perform before moving to the next level. 

In 1994-95, these learner outcomes and observable student behaviors were being used to 
develop High Expectations Learning Plans for Students (HELPS) study units recommended by the 
National Alliance for Restructuring Education. HELPS units require teachers to develop specific 
activities, student products, and assessment tools for each NCTM concept area (such as geometry 
and spatial sense or fractions and ratios — these match the 22 -day instructional rotations in 
mathematics at Ninos Bonitos). For each activity, teachers must specify the knowledge they 
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expect students to gain (the “learner outcomes”) from the activity and the skills they expert 
students to demonstrate (the “observable student behaviors”). Each instructional rotation ends 
with a “culminating activity” that requires students to employ most of the knowledge and skills 
they have learned. For instance, in the middle wing classroom observed, students created three- 
dimensional castles built of basic geometric shapes as a culminating activity. Students had to 
provide formulas for the peiimeter, area, ^nd volume of their castles in a computerized report and 
prepare an oral presentation on their project. Teachers interviewed commented on how much they 
felt they were gaining from the HELPS approach this year, and how well they felt the HELPS 
philosophy matched their own “homegrown” attempts at reform in mathematics. 

At the time of the first visit a mathematics portfolio was being planned, and by 1994-95 
mathematics portfolios had been introduced at all grade levels, although in differing forms and at 
differing paces. The movement toward portfolios is led by two upper wing teachers who are 
piloting NSP portfolios and sharing their knowledge about portfolio procedures with their 
colleagues at Wednesday in-service meetings. Ideally, teachers are to follow the NSP model for 
portfolios, which asks students to choose works demonstrating understanding of each major 
mathematical content area and to justify their choices through letters of introduction to the reader. 

In the upper wing, students are asked to choose a “favorite problem or project” from each 
rotation and to write about why they enjoyed it and what they learned from it. In the middle 
wing classroom observed, students are asked to write periodic entries in a “mathematics journal” 
for the same purpose. Some teachers, however, are using the mathematics portfolios simply as a 
holding file for all student work, making little attempt to select exemplary pieces or to ask 
students to reflect upon them. The middle wing teacher said she uses her students' portfolios for 
two purposes: to support her choice of scoring rubric for each student and to provide concrete 
evidence of student progress in parent -teacher conferences. 

Assessment Quality and Consequences 

During the 1994-95 school year, no formal evaluations of Ninos Bonitos' performance 
evaluation system have been undertaken. However, the school's staff continues to express the 
conviction that their system of measuring student performance has more validity than traditional 
kinds of evaluation. Staff members maintain that the portfolios and assessment tasks they have 
enumerated provide more complete information about the exact nature of student competency. 
They also maintain that their locally designed assessment mechanisms would have enhanced 
student performance on the CLAS. Unfortunately, as the new principal pointed out (and as noted 
earlier), the ASAT is likely a poor measure of student achievement at Ninos Bonitos, since it tests 
fact-based knowledge and skills rather than students' understanding of concepts and problem- 
solving abilities. Nonetheless, Ninos Bonitos’ reforms seem to have had a salutary effect on 
student ASAT scores, since in the second year of the study Ninos Bonitos was one of three 
schools in the district to receive state-wide recognition as a “Chapter 1 school,” based on its 
relatively strong ASAT scores. 

In order to build greater inter-rater reliability into their assessment system in language 
arts, the teaching staff last year developed portfolio anchor papers to support each rubric. Those 
papers provide a staff-approved exemplar of student work for each rubric at each language 
development and grade level. In addition, teachers note that the rubrics are so specific as to leave 
little room for major judgmental differences. At their regular staff meetings, teachers have held 
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informal “scoring conferences” to test this theory and have found high levels of agreement among 
readers about language arts scores student work should receive. In 1994*95 the new principal, 
too, was enthusiastic about the reliability of rubric scoring, noting that he raised questions about 
less than 10 percent of scores aftei reviewing the student portfolios to which they referred. 
However, as of 1994-95 no anchor papers had yet been developed for mathematics, although 
teachers engage in informal consultations about appropriate scores for student work. One teacher 
noted that she will not have complete confidence in the reliability — and hence, fairness — of her 
mathematics rubric assignments until mathematics anchor papers are developed — perhaps as 
early as the 1995-96 school year. 

The San Diego Unified School District has demonstrated its faith in Ninos Bonitos' 
assessment system by authorizing the use of language arts rubrics in a narrative report card that 
substitutes for the standard grading system. Furthermore, by the second year of the study the 
school had received a waiver to use the new mathematics assessment matrix as a substitute for the 
mathematics portion of the standard district report card. 

In addition to reflecting student work, student improvement on portfolio writing samples 
completed throughout the year are used as one criterion for judging staff performance (along with 
a review of student growth records, checklists, classroom observations, and assessments of teacher 
participation on governance committees). Ironically, California prohibits school administrators 
from evaluating teachers on the basis of standardized test results, but there is no such prohibition 
against the use of alternative assessments as evaluation criteria. In support of this use of 
alternative criteria, the Vice Principal maintains that she can identify signs of poor teaching more 
readily through a review of student portfolios than she can from a review of student grades. In 
1994-95 the new principal, too, was enthusiastic about the use of portfolios to measure the actual 
student progress that is taking place in each classroom. 

Several teachers noted that rubrics have some drawbacks; for instance, students often drop 
back several rubric levels when moving from one wing to another, which (erroneously) suggests 
to students regression rather than progression. Secondly, it is often very difficult for non-English- 
speaking students to attain the highest levels at any wing, although they may achieve tremendous 
growth within a rubric level. One teacher noted that portfolios are helpful in demonstrating this 
kind of “within rubric” growth to both parents and students. 

Resource and Training Support 

As mentioned previously, Ninos Bonitos' principal initiated the restructuring effort that led 
to the development of the school's performance assessment system, and educational experts were 
not called in until the process was well underway. Teachers noted that Ninos Bonitos was the 
first school in the district to undergo significant curriculum and assessment .eform, so that, 
initially, not much help was available, particularly in language arts. As a result, teachers said that 
they always have received the greatest assistance and support from their own colleagues at Ninos 
Bonitos. The district, however, has provided a great deal of encouragement and some regula'ory 
flexibility, especially in recent years, to support the school's continuing reform efforts. The 
County Office of Education has also provided some technical assistance, particularly when the 
district cannot provide the necessary staff development expertise. Both the district and the county 
took a great interest in Ninos Bonitos’ efforts because they felt that the success of the CLAS 
system would soon propel other San Diego schools in the direction of performance-based 
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assessments. Although the CLAS system was dismantled after the first year of study, SDUSD is 
still heavily committed to the notion of curriculum and assessment reform. 

In 1990, the RJR Nabisco Foundation awarded Ninos Bonitos Elementary a three-year 
Next Century School grant of $550,000 to improve and expand the school's reform process. This 
and an innovative technology grant from the state of California awarded in 1992 have allowed the 
school to perfect the language arts portfolio process and to record oortfolios electronically to 
accompany students' growth records. 

The teachers interviewed, including the school's union representative, felt the release time 
and training they received were quite adequate to support implementation of the school's new 
assessment system. Every other Wednesday afternoon is used for planning, for completing 
students’ growth records, and for recording portfolios electronically. In addition, teachers have 
managed to reserve three 45-minute preparation periods for themselves by cooperating in teaching 
one another’s classes. Teachers noted that fear of technology made them initially reluctant to use 
computers to electronical’;’ record student portfolios, but that the “crisis period of change” has 
passed. 



Because budget constraints do not allow the district to provide extra financial support for 
Ninos Bonitos' efforts, the school's grants committee has successfully sought outside monetary 
assistance. As a National Alliance school, Ninos Bonitos received an Apple Classrooms of 
Tomorrow (ACOT) grant that has supplied four large hardware systems and ongoing teacher 
trainim* for four upper wing and one middle wing teacher to support the maintenance of electronic 
portfolios and the use of technology in classrooms. The National Alliance affiliation also 
provides other resources, including staff development opportunities. Two teachers who sit on the 
mathematics curriculum committee, for example, were sent to Harvard's PACE summer 
mathematics training program. Others have attended the major National Alliance conferences that 
have been held jointly with NSP conferences each year at different sites across the country. 

Interaction with Other Assessments 



Teachers pointed out that six years ago, when they began their language arts reforms at 
Ninos Bonitos, very little was available in the way of national, state, or local models to inform 
their efforts. However, in mathematics, they feel they have not had to start as completely “from 
scratch,” because more national and local expertise is available to guide them. For instance, by 
the time Ninos Bonitos became an NSP partner school, language arts rubrics and portfolios were 
already in place. On the other hand, NSP, NCTM, and National Alliance materials have assisted 
teachers a great deal in developing mathematics rubrics, portfolios, classroom projects and 
activities, and authentic assessments. This year, particularly, Ninos Bonitos' teachers have begun 
to use NSP materials as a “mirror” through which to evaluate their own efforts against those of a 
highly respected national movement. 

At the time of the first visit, Ninos Bonitos planned to administer the new performance- 
based CLAS assessment in 1994-1995 — in mathematics and language arts at grade 4 and in 
social studies and science at grade 5. 

The 1993 CLAS mathematics pilot assessment results from Ninos Bonitos showed that 
none of the 54 students whose assessments were scored were rated above the mid-category of the 
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6-category performance level scale, with more than half being classified at the lowest performance 
level. On the other hand, in reading-literature, of the 45 students with scored assessments, 2 
percent were classified in the second highest performance level (5th level) and only 9 percent 
were classified in the bottom performance level. 

The CLAS assessment was to be scored by rubric, just as is student performance in Ninos 
Bonitos' language arts and mathematics curriculum. Like SDUSD's district assessment specialist, 
Ninos Bonitos' administrators said that they and other teachers were relatively pleased with the 
CLAS, since it matched the school's philosophy of providing authentic assessments and grading 
criteria to drive improved instruction in the classroom. They believe that Ninos Bonitos' reforms 
will continue to bear fruit in the form of improved student ASAT and Aprenda scores, but they 
are concerned that the these standardized tests do not accurately measure the what students know 
and are able to do as well as do performance-based examinations like the CLAS. 

Impact of Ninos Bonitos Performance Assessments 

The impact of performance assessments on the Ninos Bonitos Community is discussed 

below. 

Impact on Teachers 

Although Ninos Bonitos' new curriculum and assessment system was primarily focused on 
student benefits, the reform effort was also intended to boost “what was perceived at the school to 
be low staff morale.” Ninos Bonitos' teachers wer>’. provided with a number of incentives to adopt 
the new system, not the least of which was their full ownership of its development and 
implementation. In addition, the new configuration of classes reduced individual class size and 
provided teachers with more weekly preparation time (teachers only teach one class during the 
afternoon rotation period). Some of the numerous grants the school has received also allow 
teachers more release time and more funds to participate in staff development programs. 
Throughout the districts, six to eight days are reserved for staff development. During those days, 
the full Ninos Bonitos staff meet to work on projects such as the development of standards, 
rubrics, and portfolio assessments. In addition, every other Wednesday is a minimum attendance 
day for students at Ninos Bonitos, in which committee work, including assessment work, is 
completed. 

The Principal noted that the new system has dramatically improved classroom instruction. 
Most teachers have moved away from the traditional teacher-centered approach to instruction and 
provide, instead, more cooperative and experiential learning opportunities that focus on problem- 
solving. One teacher noted that these new approaches, such as portfolios, greatly improve her 
analysis of students' real progress and achievement levels. 

New instructional and assessment methods have also, as previously mentioned, reduced 
discipline problems and provided teachers with a greater sense of accomplishment, which is 
augmented by the knowledge that they are personally responsible for the new system's success. In 
addition, according to their union representative, teachers feel the portfolios make communication 
with parents much easier. Ninos Bonitos' annual staff surveys demonstrate that, overall, teachers 
a.e very pleased with the new system and its results. 



The overall “costs” of the system are borne, unfortunately perhaps, exclusively by Ninos 
Bonitos staff. Development of the rubrics and the portfolio system required a tremendous amount 
of work and coordination, although staff were allocated a fair amount of staff development time to 
accomplish their purpose. In addition, portfolio scoring and rubrics assignment is generally a very 
time-consuming process. Also, it is apparent that there was a great deal of resistance to the 
implementation of the system to record portfolios electronically, since staff were required to both 
learn a great deal of software and to find time outside of class to scan their students' work into the 
computer system. This latter problem has been somewhat alleviated by asking classroom aides 
and computer support staff to scan the work; complaints also have decreased as teachers have 
become “more comfortable with the system.” 

The Vice Principal noted that performance assessments place additional burdens on 
teachers since they often must meet the demands of both old and new systems. For example, in 
the first year of the study, teachers had to complete both the new mathematics assessment scoring 
matrix and the mathematics portion of the traditional report card (until the district approved the 
exclusive use of the matrix). 

The 1993-94 principal noted that although the local teacher's union pays “lip service” to 
the need for instructional and assessment reform, it has in fact presented the greatest barrier to its 
implementation, since the union allows the Principal no flexibility in hiring teachers who are 
committed to or trained in the use of the new system. 

Impact on Students 

Teachers and administrators included in the study feel that curricular and assessment 
changes have had “a palpable effect” on students. Both teachers and parents confirmed, for 
example, that children are completing fewer worksheets and using more manipulatives in their 
courses. In addition, both teachers and students commented upon the large amount of writing 
students were doing, both in language arts and in mathematics. One teacher noted that although 
nev methods may slow the pace of instruction a bit, they reinforce the fundamentals and 
foundations of mathematics and language arts, thereby promoting academic success in the long 
term. 



The teaching methods that the use of rubrics and portfolios support provide students with 
a sense of purpose, accomplishment, and enjoyment. As a result, teachers experience far fewer 
discipline problems (only one or two suspensions a year) and a much greater degree of order in 
their classrooms. The principal has personally witnessed an increase in student engagement and 
an improvement of “on-task behavior” in her recent classroom observations. Furthermore, student 
attendance rates have improved dramatically since 1988 and, at 96 percent, attendance at Nincv 
Bonitos is now ranked the best in the district. 

The parent included in the study noted that portfolios allow children to see more easily 
their own improvement. That parent feels, therefore, that students are motivated to strive harder 
than they did without the portfolio system. Moreover, she feels that weaker students do not 
experience the same sense of failure they might with standard letter grades. The vice-principal 
pointed out that the portfolio system “builds greater interaction” between the teacher and child as 
well as a greater sense of mutual respect, particularly for LF.P students from other cultures who 
find it quite a struggle to express themselves in the traditional classroom format. Almost every 
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teacher interviewed commented upon the improvements in self-confidence and motivation they 
have seen among students since the reforms began. 

These improvements in student attitude and motivation can be objectively measured, and 
are reflected in student performance, particularly in the areas of oral and written language 
development for LEP students. Anecdotal evidence further suggests that Ninos Bonitos graduates, 
particularly Chapter 1 students, are performing much better at Montgomery Junior High School, 
than they used to. Improvement in ASAT scores, however, was spotty between the two years of 
the study — at some grade levels, scores were up substantially in both reading and mathematics, 
while at other grade levels, scores were virtually unchanged or down. 

In 1994-95 the new principal prepared a detailed analysis of ASAT scores by grade level, 
recording strengths and weaknesses in each subject area. He hopes teachers will address these 
strenghts and weaknesses in their classrooms. For instance, in reading comprehension in the 1st 
grade, students were strong in “predicting outcomes,” “drawing conclusions,” and “matching.” 

A parent said that her former Ninos Bonitos child was more advanced than his peers in 
both language arts and mathematics at his new junior high school Finally, on last year's pilot 
CLAS administration, Ninos Bonitos students scored much higher in language arts than the state 
average (but much lower than the state average in mathematics, a fact that teachers feel supports 
both the success of their language arts program and the need to revise their mathematics 
curriculum). 

Impact on Parents 

Although many of Ninos Bonitos' parents are not literate, they are, according to teachers 
and administrators, very supportive of the education their children receive under the new system. 
Because parents at Ninos Bonitos are culturally and linguistically diverse, the school makes 
special efforts to involve them, providing monthly parent outreach evenings and ensuring that 
translators are present at parent-teacher conferences whenever needed. These interactions with 
parents have introduced them to the reforms at Ninos Bonitos while they are occurring. This 
year, for instance, one parent outreach evening centered around the topic of mathematics 
portfolios and another covered language arts portfolios. 

The parent interviewed in 1993-94 (also the school's parent and volunteer coordinator) said 
that parent reaction to the portfolio and rubric system is extremely positive, as measured by both 
the annual parent surveys during 1992 and 1993 and the feedback received at parent outreach 
meetings. The parent interviewed in the second year of the study reiterated this opinion. Both 
parents noted that the portfolios give parents a much more visual and concrete sense of their 
child's progress, particularly when they have difficulty reading the English language. They added 
that although some parents initially miss the letter grade system, most adapt to the new rubrics 
quite easily and eagerly (although some teachers felt that the rubrics are too complicated for non- 
English-speaking parents to understand). The parents said that the rubrics and portfolios provided 
them with much more guidance about the ways they can help their children at home with 
language arts development. Currently, both language arts and mathematics portfolios are used in 
triennial parent-teacher conferences to provide concrete demonstrations of student growth. 
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Future Plans 



Ninos Bonitos' 1993-94 principal said that her goal for the future was to continue to drive 
changes in instruction through changes in assessment, hopefully in new areas of the curriculum, 
including science. She desires to continue to convince parents and the general public that 
portfolios are valid indicators of success and to involve those parents and the public in the 
developmental process itself. 

In 1994-95, Ninos Bonitos' new principal conducted a “transition” workshop with the 
school's teachers to determine what the continuing focus of the school's reform efforts should be 
and how fast the pace of reform should continue to proceed. The principal said that he learned 
through this process that Ninos Bonitos' teachers were “spread very thin” and were not eager to 
take on additional new reforms this year. As a result, the principal's efforts this year have been 
directed at consolidating and strengthening existing innovations, such as refinement of the 
language arts rubrics and portfolio system and development of mathematics anchor papers to 
support the new mathematics rubrics. Both he and Ninos Bonitos' teachers eventually hope to 
adapt their system of performance assessments and scoring rubrics to the discipline of science — 
perhaps as early as next year. 

The school board member interviewed in 1994-95 said that, surprisingly, there has been 
little district political opposition to performance assessments and other educational reforms; she 
anticipates that, with the continued strong support of the new superintendent, the district as a 
whole will steadily expand the use of performance assessments. 



Conclusions 

Ninos Bonitos' student population is unusual in that the vast majority of its students are 
both economically disadvantaged and limited English-language speakers. Centered around written 
and oral portfolios and specific learning outcome tasks, Ninos Bonitos' performance assessment 
system has been designed to address the special needs of its particular population. Performance 
assessments have been focused initially on language arts and are used to monitor student reading, 
writing, and oral lan t 'age development. Mathematics performance assessments are in 
development — and similar systems in other disciplines are being contemplated. 

As of this writing, no objective data has been gathered concerning the validity or 
reliability of Ninos Bonitos’ performance assessments, but staff have received outside “validation" 
by consulting assessment experts on a regular basis and by developing procedures that foster 
reliability (such as anchor papers). 

Although no formal evaluation of Ninos Bonitos’ assessment system has been performed, 
anecdotal evidence from staff and parents suggests that the system's benefits outweigh its costs in 
terms of improved staff morale, better instructional practices, a more open classroom atmosphere, 
and increased student motivation. At least in language arts, this evidence is supported by 
objective measures of student performance that show encouraging progress. 






III- 15 



32 



Overall, the future of Ninos Bonitos' performance assessment system appears bright, since 
the school possesses strong leadership and has received support and accolades from the San Diego 
Unified School District. 
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EARLY CHILDHOOD RUBRICS LISTINGS 

Revision 8/1 3/92. 5/28/33. 10/93,9/21/94 



ORAL LANGUAGE 



• dots net yet moon* verbaly 

• nwwtttf evidence stfs. use* gestures and »mp*e words 

Limited Llet utf.g— «to 

• one word readme. iwmng Anns ta picture 



• rany contnbtflw m gm» settings 



• otacnoas ptcturtt using phfim two*mna word responses and/or simple sentences 

• occasionally co ntntertas m smw group sailings 

Canahte Ltetenar-Baaafcar 

• uses comoMM sa mancas that tai about a petura 

• occasionrty eontnbuiat m iarga group samngs 

Strong Lliloner-Saflikir 

• usas story-*** sentences describing ma beginning middle, and *nd of an event 

• Uses descriptive words 

• consistently contributes m class activities with relevant responses 



READING 

Iflaanartenead Raider 

•exniwa umitad attention span and tittle or no experience witn boons 

Eartv Emergent Rtidar 

• nas difficulty grasping pnnt individually 

• rents on teacnar to read stones aioud 

• may stu* be unaware mat meaning comes from text 

• parbopattf m snared reading 

Emxraam Reader 

• nas growing iWMies to use picture dues rwmory and oattems to gam -leaning from me text 

Baainninn Raider 

• is devaioomg fiuencv and reads some books witn confidence 

• a We to apply pnontes knowledge to decode 

• is mostly comtortitte reading fa/mhar text witn snort, simple narrative 

Moderately Emartancad Reader 

• feels comfomme witn boons 

• nas confidence reading text mdeDenaentty 

• cnooses to read tor pleasure 

Early Childhood Rubric 

WRITING 

Eiktttiiu 

• commumatn througn tcnMfcng ins pictures 

Emarnant Writer 

• random letters witn drawings 

• awe to distinguish between drawings and witting 

• some MOfnci of letter-like mans 

Developing Writer 

• randomly copies words from environment 

• demonstrates in unoeraunomg mat oru language can be written sown 

Beoinninn Writer 

• beginning to make coundtvmow correspondence 

• writing may include familiar words 

Eaoeriencad Writer 

• witung to invent sowing for unknown words 

• beginning to write sentences which mcJude key words to communicate mougnts let et fd - cat eat food) 

Exceptional Writer 

• cnooses lo write independently with in tbii'fv tc speii some words in a conventional way 

• writes complete sentences with Punctuation 

• snows more consistency in sentence structure spelling ano vocabulary 
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APPENDIX B 



Language Arts Growth Record 



LANGUAGE ARTS GROWTH RECORD 
Early Childhood 



I Name 



Language Placement 



Room 



ORAL LANGUAGE 
Rubrics 

Silent/Emergent Listener-Speaker 
Limited Listener-Speaker 
Developing Listener-Speaker 
Capable Listener-Speaker 
Strong Listener-Speaker 



> 

o 

Z 



(0 

2 



a> 

c 

3 



Work Habits 

Needs Motivation 
Moderately Motivated 
Motivated 



> 

o 

Z 



© 

2 



© 

c 

3 







READING Formal/Informal 
Rubrics 

Inexperienced Reader 
Early Emergent Reader 
Emergent Reader 
Beginning Reader 
Moderately Experienced Reader 



> 

o 

Z 



« 

2 



© 

c 

3 

—3 



Work Habits 

Needs Motivation 
Moderately Motivated 
Motivated 




WRITTEN LANGUAGE > % c 



Rubrics 

Pre-Writer 
Emergent Writer 
Developing Writer 
Beginning Writer 
Experienced Writer 
Exceptional Writer 

Teacher Initial 
Date 



Z 2 ^ 




Work Habits 

Needs Motivation 
Moderately Motivated 
Motivated 



> 

o 

Z 



© 

2 



© 

c 

3 

-3 




>r Transfer Purposes. 



Grade 



3 
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Development Rubrics in Mathematics 



Ninos Bonitos 
Elementary School 

A Next Century School 



DEVELOPMENTAL RUBRICS IN MATHEMATICS 

EARLY EMERGENT MATHEMATICIAN 

- Communicates/demonstrates little or no understanding of mathematical thinking 
or mathematical ideas; demonstrates little progress toward accomplishing 
mathematical tasks. 

EMERGENT MATHEMATICIAN 

- Communicates/demonstrates partial understanding of mathematical thinking ard 
mathematical ideas. Work may be incomplete or misdirected. Tools and techniques 
are rarely used. 

DEVELOPING MATHEMATICIAN 

- Communicates/demonstrates some understanding of mathematical thinking and 
mathematical ideas. Omissions in conceptual understanding are evident; however, 
demonstrates the use of mathematical tools and techniques. 

MODERATELY EXPERIENCED MATHEMATICIAN 

- Ccmmumcates/demonstrates an understanding of essential mathematical thinking 
and mathematical ideas including appropriate representations (i.e.; words, dia- 
grams. graphs, pictures). The work is usually correct and exhibits the appropriate 
use of mathematical tools and techniques, cooperates with peers and adults. 

EXPERIENCED MATHEMATICIAN 

- Communicates/demorstrates a sound understanding of essential mathematical 
thinking and mathematical ideas including appropriate representations (i.e.; words, 
diagrams, graphs, pictures). The work is usually correct, complete and consistent. 
May demonstrate the capacity to make generalizations and connections through 
multiple or unique approaches supported by effective arguments using appropriate 
mathematical tools and techniques. 

EXCEPTIONAL MATHEMATICIAN 

- Communicates/demonstrates an in-depth understanding of essential mathematical 
thinking and mathematical ideas including appropriate representations (e.g.- words, 
diagrams, graphs, pictures). The work is complete and consistent and demon- 
strates the capacity to make generalizations and connections through multiple or 
unique approaches supported by precise logical arguments using appropriate 
mathematical tools and techniques. 



Applications Assessments: 
Westgate Middle School 
June 6-7, 1994 
February 16-17, 1995 
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APPLICATIONS ASSESSMENTS: 
WESTGATE MIDDLE SCHOOL 



Introduction 



The Prince William County (Virginia) Public Schools, as part of an ongoing district-wide 
restructuring effort, introduced its Applications Assessments in mathematics, science, and language 
arts during the 1993-94 school year. These assessments are just one piece of the school district's 
restructuring effort, which is guided by a district-developed restructuring process, including a 
Quality Management Plan, newly adopted Standards of Quality, district-wide school-based 
management, and other related elements. 



The district's work toward a reformed educational system has progressed rapidly. During 
the summer of 1994. Pnnce Wiliiam County teachers worked to develop performance standards 
for the new Applications Assessments. In addition, teacher teams revising the district’s curricula 
completed their work (in most subject areas), and the new' curricula were adopted by the school 
board. Finally, the district, partially in collaboration with Riverside Publishing Company and 
partially on its own. developed a fourth Applications Assessment in the area of social studies: this 
assessment was to be administered for the first time in Spring 1995. Thus, the Prince William 
County Public Schools provide an example of a district struggling simultaneously with the many 
pieces of a comprehensive school reform effort. 

Westgate Middle School, one of 12 middle schools in the district, was visited twice as a 
part of this study, first in June 1994 and again in February 1995. The school served 827 sixth, 
seventh, and eighth grade students during the 1993-94 school year. These students were, on 
a\erage. somewhat wealthier than the district average, and the school had a smaller percentage of 
minority students than did the district as a whole. Westgate students were mostly white (80.7 
percent): other students were African-Amercan (10.6 percent). Hispanic (4.8 percent). Asian/ 
Pacific Islander (3.7 percent), and Alaskan/Native American (0.1 percent). About 8.7 percent of 
students qualify for the 11. S. Department of Agriculture's free or reduced price lunch program, a 
lower percentage than the statewide percentage of students — 12.3 percent — eligible for Chapter 
I services. (These statistics remained essentially unchanged for the 1994-95 school year.) 
Westgate's faculty includes ft 5 teachers, two administrators, six teaching assistants, and 25 
classified personnel. 

Participants 

Numerous individuals were interviewed for this case studs during the two site visits. 

Their roles are illustrated in I xliibit I. 
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EXHIBIT I 



Study Participants 



1993-94 


1994-95 ] 


• Supervisor, Prince William County Public 
Schools' Office of Assessment and 
Evaluation 

• Assistant Principal. Westgate Middle 
School 

• Reading Specialist and Assessment 
Coordinator. Westgate Middle School 

• Fourteen Teachers. W'estgate Middle 
School — 

- Eleven seventh grade teachers of math, 
language arts, social studies, science 

- One eighth grade teacher of English 

- Two special education teachers — 
hearing impaired, emotionally 
disturbed 

• Two Special Education Teaching 
Assistants. W'estgate Middle School 

• Two Parents of Westgate seventh grade 
students 

• Sixteen Westgate Middle School seventh 
grade students — 

- T hirteen regular education students 

- Three special education students 
(hearing impaired), mainstreamed for 
part of the school day 


• Supervisor. Prince William County Public 
Schools’ Office of Assessment and 
Evaluation 

• Reading Specialist and Assessment 
Coordinator, Westgate Middle School 

• Ten Teachers. Westgate Middle School 

- Seven seventh grade teachers of math, 
language arts, social studies, science 

- Two eighth grade teacher of English 

- One special education teacher — 
hearing impaired 

• One Teacher, member of district’s 
Instructional Support Team 

• One Parent of Westgate students 

• Twelve W'estgate Middle School eighth 
grade students 



Observations 



In I clvuarv l ( W5. a team meeting of one of the two Westgate Middle School seventh-grade 
teaching teams was observed. Addilionailv . the science classes taught b> one of these teachers 
were also observed. 



District Context 

Prince William Counts is a suburban and rural counts in Northern Virginia, located not far 
from W ashington. D C Main residents commute to the Distnct. but a good portion of the countv 
is rural and far from casv commuting range. The area includes both historical landmarks 
several Civ il W ar battles were fought here and extensive outlet shopping malls. 
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The Prince William County Public Schools (PWCPS) operate 39 elementary, 12 middle, and 
7 high schools throughout the county (the district serves most, but not all. parts of the county; a 
few small school districts serve the children of some Prince William County municipalities). In 
all, the district served over 45,000 students during the 1993-94 school year. This student 
population is approximately 72.0 percent white, 18.8 percent African-American, 5.2 percent 
Hispanic. 3.6 percent Asian-American. and 0.4 percent from other racial and ethnic backgrounds. 
Just over 16 percent of the district’s students qualify for a free or reduced-price lunch. These 
statistics remained essentially the same for the 1994-95 school year. 

Since the late 1980s. the school district has been planning and implementing a comprehensive 
restructuring effort encompassing (a) development and adoption of the district’s Quality 
Management Plan, (b) articulation of a set of expectations for student achievement entitled 
Standards ot Quality, (c) expansion of the district's Assessment Program, and (d) revision of the 
curriculum for all subjects and at all grade levels. 

I he district's work in these areas is still in midstream, and the restructuring effort, as 
currently planned, will not be complete for several more years. 

Purpose of the Applications Assessments 

I he Prince William County Public Schools identified a need for a new assessment tool to 
accompany the district's evolving educational objectives. As part of the district’s Quality 
Management Plan (see below ), the district adopted six Standards of Quality, as summarized in 
Exhibit II. The primary purpose of the Applications Assessment is to help the district measure 
the progress students make tow afa the attainment of the new standards. 

I \\ C I S hope to meet at least two additional objectives with the Applications Assessments, 
first, the assessments will eventually be used for accountability purposes in schools and 
classrooms, though the district has no plr.ns to introduce rewards or sanctions for performance. 
However, data will be maintained and reported at the student, classroom, school, and district 
levels. Second, the Applications Assessments are intended to communicate to teachers, parents, 
students, and the community what students know and are capable of doing. In this sense, the 
Applications Assessments join othei facets of the district's assessment program to monitor overall 
student achievement and the quality of the district's educational program. 

Although in the future the Applications Assessments will support all of the above purposes, 
the first full-scale administration in Spring 1994 was intended to collect baseline data about 
student achievement at grades three, seven, and ten. 

Development of the Applica t ions Assessments 

!’\\( TS' Applications Assessments ate administered to third, seventh, and tenth graders. I he 
Assessments in mathematics, science, and language arts were piloted in November 1993 and 
administered district-wide for the first time in May 1994. An Applications Assessment in social 
studies was developed during the summer and fall of 1994 and was to be administered for the fo>t 
time in Spring 1995. 



EXHIBIT II 



Prince William County Public Schools’ Standards of Quality 1 



Students should be: 

• Knowledgeable and proficient in the traditional basic academic skills 

• Good thinkers, problem solvers, and decision makers 

• ilffective communicators 

• l sers of technology 

• Knowledgeable of various racial and ethnic cultures, as well as differences based on gender, 
age. and physical ability 

• Good citizens 



When the district made the (tension to include a performance assessment as part of its 
Assessment Program, its administrators sought a test publisher that could fulfill the following 
criteria: 

• "Pencil and paper" assessments — administration had to be straight- forward and not 
overlv time-consuming; 

• Inclusion in the assessments of higher-order thinking skills and linkages to standards 
established b\ such leading organizations as the National Council of Teachers ot 
Mathematics and the Association for Supervision and Curriculum Development; 

• Comparability of results to national, normative data; and 

• Use of a coherent theme carried throughout the assessment; for instance, on the language 
arts assessment, the questions to which students respond might all relate to one story, 
though questions might call for responses employing different types of writing. 

C ontracting with the Riv e rside Publishing Company 

lo develop the first three assessments (in math, science, and language arts). PWCPS 
contracted out with the Riverside Publishing Company. In the words of the district's Supervisor 
of the Office of Assessment and {‘valuation. "Riverside was the only test publisher that seemed to 
m et our objectives " PWCPS contracted with Riverside to produce two forms, form A and 
form B. of each assessment. The district will administer the two forms in alternate years (i.e.. 

I orm \ m IUU5. form B in l*>%. form \ in l*W7. and so lorth). 



1 Prime W illiam (.'minis Public Schools. Vn^ilih \hintwmcm Plan, as revised January N IW 
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The Applications Assessments are modeled on the Riverside Performance Assessment Series 
(R-PAS). During the summer of 1993, 180 Prince William County teachers — representing 
grades three, seven, and ten and language arts, science, and mathematics — met to review and 
revise potential performance assessment tasks supplied by Riverside. They retained many of the 
supplied items and made recommendations for revising others. The suggestions made by these 
teachers led to some changes in assessment items (though not prior to the pilot test of the 
assessment). 

During the summer of 1994. PWCPS began to work with Riverside to select assessments in 
social studies for grades three, seven, and ten. The district’s administrators and teachers felt that 
the assessments Riverside offered for grades three and seven were appropriate for the district's 
students and curriculum but that no assessments available through Riverside were appropriate for 
the district’s tenth graders. Consequently, the district opted to develop its own Applications 
Assessment for tenth grade social studies. During the 1994-95 school year, teacher volunteers 
worked to develop the assessment. Riverside is assisting the district in its work by critiquing the 
assessment: ultimately. Riverside will help the district revise and package the assessment. 

Pilot Tests 



The f orm A Applications Assessments in math, science, and language arts were piloted in 
November of 1993. All high schools and middle schools and about half of the district's 
elementary schools participated in the pilot test; approximately 400 students at each grade level 
were included in the pilot. (Students participating in the pilot were fourth, eighth, and eleventh 
graders, as these students had only recently completed the grades at which the district administers 
the assessments.) The pilot test was scored by two corporations hired by Riverside Publishing 
Company . 

During the winter of 1994. the district reviewed the results of the pilot test with teachers. 
Additional assessment items also were evaluated, and. again, the district pro ided Riverside with 
recommendations for modifications prior to the spring administration of the •■•st. 

In November 1994. Form B Applications Assessments in math, science, and language arts 
were piloted. In addition, the new Applications Assessments in social studies were piloted for 
grades three and seven. The tenth-grade social studies Applications Assessment was to be piloted 
later in 1 995. 

Characteristics of the Applications Assessments 

I lie \pplications Assessments constitute a oettormancc event during which students must 
construct responses to questions and prompts in the areas of language arts, mathematics, and 
science During the Spring 1994 administration of the exam, each student took only one of the 
three sections. For the Spring 1995 administration, all students were going to be required to take 
the math and the language arts assessments. In addition, the district was going to administer the 
science and social studies assessments to a sample of 25 percent of students (25 percent of 
students were to take the science assessment. 25 percent the social studies assessment, and the 
remaining percent would take neither! 
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Format and Time Limits 



Each assessment is contained within a booklet. A typical assessment has about a dozen items 
which call for anything from a phrase to a multiparagraph or multistep response. In 1994 students 
were allowed up to two hours to complete the exam, although most students reportedly took much 
less time to finish, and administrators have called the exam untimed. For the Spring 1995 
administration, the assessment will be essentially untimed: that is, students actively working to 
complete the assessment may have as much time as they require. The assessments, however, will 
still be designed to take most students about two hours to complete. 

Scoring Rubrics 

Scoring rubrics are used on each item on the Assessments and are tailored to the individual 
task: the rubric for one task might call for the scorer to rate students' responses on a 1- to 4-point 
scale, while other items might be worth only 1 point — the student either gets credit for the 
response or does not. Most rubrics for individual assessment items differentiate between 
responses on 2-. 3-, or 4-point scales. (Examples of two rubrics — one for a series of math tasks 
and one for a series of language arts tasks — appear in Appendix A.) Scoring of the Applications 
Assessments is done by an outside company associated with Riverside. At one point in time, the 
district considered having its own teachers score the assessments, but gave up the idea as 
impractical. 

Provisions for Special Populations 



Students who receive special education services take the Applications Assessments under 
modified conditions, as called for by each student's IEP (or not at all. again, according to the 
I UP). Mediated conditions include extra time, extra instructions from the teacher, dictating 
responses to an adult to write down, and assistance in reading the assessment items. 

Development of Standards of Performance for the Annlications Assessments 

Over the summer of 1994. Prince William County teachers came together develop standards 
of performance for the assessments. The approach taken by the district to accomplish this attack 
was two-pronged. First, without having any information available to them concerning actual 
student performance on the spring 1004 assessment, teachers wrote standards for satisfactory 
performance on the assessments at each grade level. Then, after reviewing information about the 
scored assessments, teachers reviewed the standards they had constructed to reconsider whether or 
not they were appropriate. 

Based upon the standards written by Prince William Counts teachers, between 40 and 60 
percent of students assessed in Spring 1 004 met or surpassed the standards for the nine 
assessments (i.e.. third-grade math, science, and language arts: seventh-grade math, science, and 
language arts; and tenth-grade math, science, and language arts). According to the Supervisor of 
Assessment and Evaluation, teachers adhered to the standards they hud written even in the cases 
where fewer than half of students assessed met the standard. 
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The district’s Supervisor of Assessment and Evaluation was, he said, very pleasantly 
surprised by how well the standard-setting process went. The process went more smoothly than 
he would have anticipated, and the standards written were of high quality. He also reported that 
teachers were equally pleased with the process and its outcomes. 

Evaluations of the Applications Assessments 

Riverside Publishing Company conducted studies of the interrater reliability of the scoring 
process following both the 1993 pilot administration and the Spring 1994 full-scale administration 
of the Applications Assessments. In both cases, they found interrater reliability to be high; for the 
10 percent sample of Spring 1994 Applications Assessments for which interrater reliability was 
computed. Riverside found it to be at about 95 percent. 

District administrators assert that the assessments were chosen by virtue of their content 
validity: the district believes they assess something valuable. However, the district intends to 
continue to review the content validity of the assessments with teachers. 

Teachers have expressed two types of concerns about the content validity of the assessments. 
First, math teachers, both at the tenth-grade and the seventh-grade levels, are concerned that the 
assessments include some material many students have not had a chance to study or to master (for 
instance, the tenth grade assessment includes some geometry, and the seventh grade assessment 
focuses on percentages). Second, the content validity of the Applications Assessments is 
necessarily ambiguous given the fact that the district selected and implemented the assessments 
prior to revising the curriculum. Thus, the relationship between the Applications Assessments and 
the revised curriculum remains to be examined. 

District administrators say they intend to bring teachers together to review these unresolved 
issues. In addition, the district intends to follow up with teachers concerning the consequential 
validity of the .sessments — that is. how teachers use the information provided by the 
Applications Assessments. These evaluations, however, remain to be done. 

Resource and Staff Development Support 

To develop the Applications Assessments. PWCPS has drawn on financial and human 
resources from both within and outside the district. In addition to supporting the development of 
the assessments, these resources hav e supported sev eral staff development activities designed to 
inform teachers about the new assessments and to solicit their reactions. Say s one teacher. "The 
district office person coming to school and giv ing us an inservice was one of the things done 
right." In addition, the district has attempted to keep the community informed about the 
assessments and their purpose. 

Finn lie iai R esources 

The development of the Applications Assessments will have cost the PWCPS approximately 
$300.00(1 over the 1 993-94 and 1994-95 school years. In addition, expenses associated with 
scoring the assessments cost $4 50 per student; as approximately I ().()()() students in grades three, 
seven, and ten take the assessments each y ear, annual scoring costs will be about $45,000. 
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Approximately 25 percent cf the cost of the program is borne by the Federal government, while 
the remaining 75 percent comes from local sources. 

Human Resources 



Many PWCPS teachers, especially those who teach third, seventh, or tenth grade, have been 
involved in the process of refining the Applications Assessments. Additionally, many high school 
social studies teachers have taken on the job of developing the tenth-grade social studies 
Applications Assessment. The involvement of the district's teachers represents both a source of 
human resources as well as staff development, as teachers learn more about performance 
assessments and return to their schools with their new knowledge. 

During the winter of 1994. the district held two two-day workshops for (primarily) third, 
seventh, and tenth grade teachers. About 70 teachers attended each workshop, and 20 more 
attended a make-up session. The workshops focused on the following: 

• An overview of the Prince William County Assessment Program and the role performance 
assessment will play in it: 

• Teachers taking the assessments to be administered to students at the grade level at which 
they taught: 

• Teachers learning how to use the scoring rubrics and scoring their own exams. As part of 
reviewing the rubrics, teachers also saw anchor papers to help identify low and high 
quality responses: 

• A presentation of results of the November pilot test, item by item; 

• Discussions of how test items could be improved in the future: 

• Discussion of how the next step in the restructuring effort — the standard setting process 
— can best be approached; 

• Planning for the spring administration of the Applications Assessments: and 

• Review of the Resource Kit (see below) to be distributed to all schools. 



Teachers who attended these workshops were charged with the task of bringing vvliat thev had 
learned hack to other teachers at their schools. 

I he district did not anticipate repeating these workshops prior to the Spring 1995 
administration of the Applications Assessments. 

Teacher Resource Kit 



The district compiled a Resource Kit. which was distributed to all schools in the district in 
March of 1994. The Resource Kit provides teachers with information about both the Applications 
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Assessments and the Basic Skills Assessments, the two new elements in the district’s Assessment 
Program. A similar Resource Kit was to be distributed to all schools prior to the Spring 1995 
administration of the assessments. 

School Focus and Videos 



Two district administrators, the Supervisor of the Office of Assessment and Evaluation and 
the Assessment Specialist, appeared on a local public television program called School Focus, in 
January of 1994, to discuss the new Applications Assessments and other elements of the district's 
assessment program. The district recorded this program on video for inclusion in the Teacher 
Resource Kit and for broader distribution to parents and community members. Similarly, district 
administrators made another video, this one concerning the district's entire assessment program — 
norm-referenced tests (1TBS), criterion-referenced tests, Applications Assessments, and Basic 
Skills Assessments — in February 1995. This video, too. was to be made available to district 
teachers and other interested individuals. 

Interaction with Other Reforms 



The Applications Assessments are only one of several reforms Prince William County is 
currently introducing into its school system. The district has adopted a Quality Management Plan, 
which delineates a plan for restructuring schools to support students' attainment of six Standards 
of Quality adopted by the district (described above). One of the six standards focuses on basic 
skills, and the district is adopting new Basic Skills Assessments to complement the Applications 
Assessments. In addition, the district is in the process of revising its curricula at all grade levels, 
in all subject areas. Finally, a state-level reform — Virginia's Literacy Passport — is a 
significant piece of the district’s Assessment Program. - 

Quality Management Plan 

The Prince William County Public Schools Quality Management Plan (QMP) was developed 
over two years, beginning in the 1991-92 school year. Borrowing the language of its exemplar. 
Total Quality Management (the plan speaks in terms of "customer satisfaction"), the plan sets out 
a vision statement, a mission statement. Standards of Quality, goals, and performance standards 
for the school district in order “. . . to define a single purpose for all employees, to focus on that 
purpose, and to continuously improve the operation of Prince William County Public Schools" 
(QMP. p. I). The QMP encompasses the school district's long-range plan, schools' annual plans, 
school district department plans, and the district budget. Adopted in June of 1993. the QMP was 
revised in September of 1993 and again in January of 1994. 



( 'lie additional element nt icstructunng initiated at the district level is school-based manauenient Ibis 
piece of the district's restructuring effort is described below because, although it was initiated at the district 
level, it is carried out primarily at the school lc'«.I 
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Standards of Quality 



One important element of the district’s Quality Management Plan is the six Standards of 
Quality, already described above. During earlier iterations of the QMP, only five standards were 
delineated. However, concern in the community over what some people perceived to be value- 
laden standards led, in January' of 1994. to the addition of a standard for students' basic skills 
attainment. The Standards of Quality are what the district's Supervisor of Assessment and 
Evaluation calls the district's "expectations" for student educational outcomes, and it is progress 
toward the attainment of these standards that the Applications Assessments and the Basic Skills 
Assessments are intended to measure. 

Basic Skills Assessments 



The second recent addition to Prince William County's Assessment Program is the Basic 
Skills Assessments. Piloted in the Spring of 1994 in conjunction with the first full-scale 
administration of the Applications Assessments, the Basic Skills Assessments are multiple-choice, 
criterion-referenced exams administered at grades 3. 5. 7. and 10. These assessments are intended 
specifically to measure student progress toward attainment of the Standard of Quality pertaining to 
“basic skills." 

Applications Assessments and Basic Skills Assessments are intended to complement each 
other and other components of the district's assessment program (e.g.. the ITBS and Virginia's 
Literacy Passport). While the Applications Assessments aim to measure students' abilities to 
apply knowledge, the criterion-referenced Basic Skills Assessments are more traditional in their 
assessment of students' mastery of subject matter. 

Curriculum Revision 



During the 1993-94 school y ear, the Prince William County Public Schools moved to revise 
its curriculum at all grade levels. Teachers interested in participating met to evaluate the existing 
curriculum and to identify desirable revisions. Groups of teachers (e.g.. fourth grade teachers, 
seventh grade science teachers) moved at different speeds, but over the course of the school year, 
most groups presented preliminary revisions of the curriculum to the group(s) they were 
representing. Teachers district-u ide were then invited to comment on the revisions. Ov*r the 
Summer of 1994. the revisions were essentially complete, and the school board formally adopted 
most ol the reused curricula. 

The implication of curriculum revision fo r the Applications Assessments is unclear. One 
teacher inters iewed for this study has been involved in the revision of the language arts 
curriculum: she suggests that the revised curriculum corresponds well to the Applications 
Assessment in language arts. However, the Applications Assessments, according to administrators 
and other teachers, did not guide the curriculum revisions in any direct way. Therefore, it 
remains tor the district's future evaluation of the content validity of the assessments (described 
above) to reveal how closely the revised curricula and the Applications Assessments correspond to 
one another 
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Virginia’s Literacy Passport 



The state of Virginia introduced its Literacy Passport in 1989. The assessment, taken by all 
sixth graders, comprises three sections — reading, writing, and mathematics. 

Successful completion of all three sections is intended to reflect students’ mastery of basic 
literacy. Students who do not pass the exam during either of the two sixth grade administrations 
may retake those sections they do not pass during their seventh and eighth grade years. Students 
who still have not passed all three sections by the end of eighth grade are prohibited from being 
considered “full" ninth graders: although they may be promoted to high school, they may not take 
part in certain extracurricular activities and elective coursework until they have passed all three 
sections of the exam. 



Because of the fairly high-stakes nature of the exam, the Literacy Passport has been 
somewhat controversial. The high stakes also mean that it is a well-known rite of passage at 
Westgate Middle School. Teachers, students, and parents alike are familiar with the exam and its 
consequences. 



School Context 



Westgate Middle School, located in suburban Prince William County, is housed in a building 
formerl). used as a high school. The school is marked by a decentralization of authority — the 
principal is a firm believer in delegation of responsibility . The school, like other Prince William 
Count) schools. Lps adopted site-based management practices. 

Since the late 1980s. seventh and eighth grade teachers have been organized into teams, each 
grade level being divided into two groups. Thus, one half of the seventh grade student body is 
taught by teachers on Team A. while the other half is taught by Team B teachers. Teaching 
teams have common planning time every other day. one group meeting on even numbered days 
and the other on odd. 

School Level Education Reforms 



In addition to the elements of education reform introduced by the district and described 
abo\e. Westgate Middle School itself has initiated or been a central player in two school-level 
reforms. I he first is school-based management, initiated by the district over the past five years 
but carried out at the school level. Second. Westgate Middle School has bc*’n introducing 
portfolio assessment school-wide during the past three years.' 



Woodhndge Middle School's use of portfolios, though an element of the restructuring taking place at the 
school, is not the focus ot this case sludy llnis. they arc described only briefly here. 



O 
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School-based Management 



Beginning in 1989. the Prince William County School District piloted and then introduced 
districtwide school-based management. Westgate Middle School is managed by a Planning 
Council comprised of 14 teaching and other instructional staff members. 6 parents, and 2 
administrators. 

All schools in Princ? William County must adopt an Annual School Plan. Westgate's Annual 
Plan for the 1994-95 school year (adopted in January 1994) sets out eight objectives to be 
achieved during the school year. One of the eight objectives is “To demonstrate growth and 
improvement based on academic achievement, surveys, reports, portfolios and standardized test 
results as measured by Iowa Tests of Basic Skills and Virginia Passport Testing** (Westgate 
Middle School. Annual School Plan. January 31. 1994. p. 21). 

Interestingly, though perhaps not surprisingly, conflicting impressions of the balance of power 
on the Planning Council emerged from individuals participating in this study. While one teacher 
was indignant that. **We allow parents to control this facility. . . . They dominate the school 
council." a parent suggested that parents have very little influence at all. 

Portfolios 



The aforementioned reforms are all district driven reforms. At the school level, however, 
another assessment reform is taking place. Throughout the school, teachers are helping their 
students to maintain portfolios of their work over the course of each school year. 

During the 1991-92 school year. interested teachers at the school, led by the Language Arts 
Department Head, researched the portfolio concept and worked on developing a method of using 
them that made sense to Westgate teachers. During 1992-93 seventh and eighth grade teachers 
used portfolios for the first time. Each team develops a portfolio system that works for them. 

Ev ery student has a faculty adv isor from within his or her team who is responsible for guiding the 
student in the selection of work to include in the portfolio; each teacher advises about 17 students. 
QuarterK. students select work they have completed for inclusion in the portfolio, conduct self- 
assessments of their work, and develop goals for the upcoming quarter. 

Westgate teachers do not score the portfolios. T hey do suggest, however, that the portfolios 
have provided valuable insights into their students' learning processes and attitudes toward school. 
I or instance, one teacher commented. "When the kids chose almost only tests to go into their 
portfolios the first quarter, it really shook us up. We realized we had to look for ways to 
encourage students to value other types of work." Additionally, teachers suggest that they have a 
broader understanding of their students than they used to have when they never saw the students' 
work in other subject areas. 

Impact of the Applications Assessments and Other Reforms 

Westgate Middle School provides an example of a school in transition. Multiple reforms are 
taking place simultaneously, and their ultimate impact — singly and cumulatively — remains to 
be seen. It is too early to discern an impact of even single elements of districtwide and 
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schoolwide reform, such as the Applications Assessments. Still, the reactions of teachers, 
students, and parents to the Applications Assessments and other reforms can be described. 

Impact on Teachers 

Teachers at Westgate Middle School fall into two camps over the Applications Assessments 
and other current restructuring efforts — the generally optimistic and the somewhat skeptical. 

Teachers’ reactions to the applications assessments . Most Westgate teachers hold 
favorable opinions of performance assessments in theory'. As two teachers commented, “It's the 
only logical way of doing assessment," and "We need to get these kids to write more. They've 
been allowed to fill in dots for too long." 

Many teachers' reactions to the Applications Assessments are also positive, though some hold 
more critical views. Teachers found that the language arts and science assessments were well 
aligned with the district's current curriculum and actual classroom experiences, and one language 
arts teacher felt the assessment was also well aligned to the new curriculum. However, several 
math teachers said that the math assessment was developmentally inappropriate for their students, 
for it tested skills most seventh grade students have not yet mastered (much of the exam focused 
upon the calculation of percentages). Teachers also expressed concern about inaccuracies in the 
math assessment: the assessment misused the words "percentage" and "percent.** for example, thus 
confusing students who were aware of the distinction. 

Some teachers expressed concerns that the science and math assessments were really reading 
comprehension tests, noting that "kids who don't read well have a problem." One math teacher 
felt that the math Applications Assessment represented an encroachment of language arts into the 
mathematics discipline: this teacher said. "If it s something that helps students learn math. I'll do 
it." but that she was not yet certain t at "all this writing in math" is beneficial to student learning. 
Another math teacher, however, expressed very positive feelings about the math assessment, 
noting that it is aligned with the standards developed by the National Council of Teachers of 
Mathematics. 

Finally, one teacher expressed a concern about the subjectivity inherent in the use of scoring 
rubrics, and another cautioned. "Performance assessment is not the end all and be all. Educators 
tend to get caught up in fads." 

Teachers' reactions t o the development process and training activities . Teachers 
applauded the district's efforts to include them in the process of developing the Applications 
Assessments. However, one teacher who had been involved in the refinement of test items 
indicated that Riverside had not incorporated the changes she arid other teachers had 
recommended for one of the assessments. Another teacher expressed dismay that the district 
wanted her to serve as a "rubber stamp;" in her opinion "the district had already chosen a testing 
company to create a test for a curriculum we hadn't chosen vet." 

Teachers who had participated in some form of inservice activity focusing on the 
Applications Assessments were universal in their praise of the format and usefulness of these 
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activities. Special education teachers and aides also reacted favorably to the inservice activity that 
focused on how they could provide accommodations for their students taking the assessments. 

Teachers’ reactions to other reforms . Most Westgate teachers expressed positive reactions 
to the curriculum revisions they had seen as of spring 1994. One language arts teacher liked the 
fact that the new curriculum placed “more emphasis on how than on what" teachers taught and 
students learned. Math and science teachers also expressed satisfaction that their curricular 
objectives were being reduced to a much more manageable number. One teacher, however, said. 
"Restructuring is not the answer. It's cyclical. We’ve been through it all before. These 
curricular objectives look a lot like the ones we had 12 years ago." 

Westgate teachers expressed one substantial dissatisfaction with the process the district is 
pursuing to implement the new standards and other reforms. Because of the reform effort, the 
district has reassigned its subject area supervisors to Restructuring Support Teams (renamed 
Instructional Support Teams in 1994-95). A direct byproduct of the expanded Assessment 
Program and other reforms, this reassignment of district personnel has left some teachers feeling 
rudderless. As one teacher put it. "Introducing a revised curriculum and eliminating our area 
supervisors at the same time is a double whammy." This widespread reaction on the part of 
Westgate Middle School teachers suggests a point of disconnect between district officials and 
teachers. Though district administrators maintain that the supervisors are still available and 
merely are wearing different hats, this suggestion is clearly at odds with teachers' perceptions, 
expressed both in 1994 and in 1995. 

Teachers' comments on feedback and communication with district . Surprisingly. 

Westgate teachers reported in February 1995 that they had not yet received feedback from the 
district imi student performance on the Spring 1994 Applications Assessments. Additionally, most 
teachers said they were unfamiliar with the standards of performance a group of district teachers 
had developed over the summer. 

Teachers were also ignorant about other facts concerning the Spring 1995 administration of 
the assessments. For instance, teachers interviewed did not know' that students would take both 
the language arts and the math assessments in 1995 and that a subset would also take the science 
or social studies assessment (in 1994 each student took only one of the assessments). 

Furthermore, teachers were unaware that the math, science, and language arts assessments to be 
administered in 1995 would be identical to those administered in 1994 (Form A was used in 1994 
and 1995; Form B will be used in Spring 1996 for the first time). (Note, however, that at the 
time of the 1995 interviews, the district had not yet distributed Resource kits for the 1995 
administration of the Applications Assessments.) 

Impact on Curriculum and Instruction 

A lew teachers identified small changes in their instructional practices made in response to 
the introduction of the Applications Assessments, l or example, one math teacher said that, 
because of the emphasis of the 1994 assessment on percentages, she had rearranged her 
curriculum so as to cover percentages earlier in the school year (she reported that last year's 
assessment was administered before she ml taught percentage concepts to students). A special 
education teacher also said that, because Thu experience administering the Applications 



I V- 1 4 




Assessments to her hearing-impaired students, she had begun to reflect upon changes she could 
make in her instructional methods to foster better language development in these children. 
However, she said she had not yet made any changes in her instructional techniques. 

Most Westgate teachers identified only modest changes in their approaches to instruction and 
assessment in their classrooms. An illustration of one teacher's classroom techniques and changes 
compatible with, though not necessarily spurred by, the Applications Assessments appears in 
Exhibit III. 



However, the predominant impact the assessments have had on instruction to date is in the 
area of test prepar‘ : on. Teachers prepared their students for the assessments in different ways. 
Some spent as muu. as three days prepping students, while others spent only a few minutes. 

Thus, the number of school hours devoted to test preparation and administration varied across 
classrooms. Several teachers purchased sample tests, paying for them out of their own pockets, to 
use with their students. 

Two 'eachers stated that the Applications Assessments will necessarily affect instruction. 

While one teacher linked the impact of the Applications Assessments on instruction to. any future 
accountability system ("When we have accountability, we will teach directly to the test.”), another 
prophesied. "If nothing else, there will be another kind of test-uking skill to be taught.” 

Impact on Students 

In 1993-94. students’ almost universal responses when asked about the Applications 
Assessments were that they were "easy and that tlvw prefer multiple-choice exams. Some 
students said they liked having the chance to explain their opinions about something (on the 
language arts exam), while others said that the themes carried throughout the exams were 
"boring. " One girl said. “They should try to choose a theme that kids will like better.” reacting to 
the math assessment s "crossword puzzle contest" theme. 

Despite students assertions that the assessments were easy for them, their teachers doubt that 
they performed well. According to one teacher. "We're going to have to lower our standards and 
acknowledge that these children have not been trained to be problem solvers. . . . Ninety percent 
ol our kids can t do this kind of test. It s an idealistic assessment. Teachers suguested that their 
students did not take the test seriously : they saw no reason to put in their best effort because they 
were not being graded on it and no score would go home. 

In |0‘)4-95. seventh-grade students had not yet taken the Applications Assessments, and the 
eighth-grade students interviewed expressed similar reactions to those shared by their classmates 
the previous year. 

impa ct on special populations . Students receiving special education services had an 
especially hard time with the Applications Assessments. As one Teaching Assistant put it. "The 
Applications Assessments focus on critical thinking skills and problem solving skills — things 
these kids usually struggle with I hey also have trouble with reading comprehension, which is 
also a problem on this kind of test." 
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EXHIBIT III 



One Teacher's Reflections Upon Education 
Reform and Modifications in Her Classroom 



Suzanne Conway runs a busy, fast- moving seventh-grade life science classroom. During five 
50-minute class periods each day. she daily makes minor changes in the day s lesson plan so that 
she can keep things interesting and test out teaching techniques that work best. 

Though Suzanne says she has not made any changes in her teaching practices as a response to 
the district's introduction of the Applications Assessments, her teaching has changed over the past 
several years in ways compatible witn the new assessments. For instance. Westgate Middle 
School's adoption of team teaching several years ago has led to increased (though. Suzanne says, 
by no means complete) integration of subject area instruction. She says she emphasizes written 
language much more in her class than she did in the past, and she also introduces students to 
multiple ways of communicating information, such as models, graphs, charts. 

Suzanne says she has used rubrics for at least 1 5 years to score students' project work. 

Howev er, students do not see the rubrics ahead of time, and Suzanne believes that she could 
improve her skills in writing and using rubrics. 

Most directly relevant to the district's new assessment is Suzanne's recent emphasis on what 
she calls "Science Starters." These activ ities, described on the blackboard of her classroom, 
emphasize science skilis (e.g.. data collection techniques) and concepts (e.g.. cellular 
reproduction), not rote memorization of information. 



Students enrolled in a class for children with hearing impairments commented that they prefer 
multiple choice tests to the Applications Assessments (most of their nondisabled peers said the 
same thing). However, those same students liked the fact that the new assessment allowed them 
to ask their teacher to provide them with extra directions. That was "a good idea." said one boy. 
The teacher of the children with hearing impairments commented that the Applications 
Assessments are not a v alid test of their knowledge because the emphasis on language makes it so 
hard for them: the students must read, and then write, making the Assessments doubly hard for 
them. Ibis teacher speculated that she would probabiv place more emphasis on the types of 
questions she uses with her students, aligning them with the Applications Assessments. 

Impact of portfoli os In both 1993-94 and 1994-95. students' responses to the use of 
portfolios at WMS are almost universalis positive. Though a few students grumbled about having 
to uo over work thev had alreadv completed, most agreed with one bov who said. "I like looking 
back at mv past work and seeing how much I've improved." 
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Impact on Parents 



The impact of the Applications Assessments on parental involvement in children's education 
is. as of yet. negligible. The two parents who participated in this study were not well-informed 
about the Applications Assessments. They were aware that their children had taken the exam, and 
they knew which of the three sections they had taken. Interestingly, neither parent believed she 
had received any information from the school about the assessment ahead of time, despite the fact 

that a notice was sent home. (One parent chalked this up to the unreliability of her seventh trader 
cum messenger.) 



One of the parents said that her daughter has significant trouble taking tests. The seventh 
grader has not yet passed all three sections of the Literacy Passport, and both mother and daughter 

are concerned about her being able to participate as a "full” ninth grader when she enters high 
school. 



Future Plans 

I lie Prince William County Public Schools intend to use the results from the spring 1994 
administration of the Applications Assessments as baseline data against which future student 
scores can be judged. The district has waffled in the past about the level(s) at which scores 
would he reported (at one point, the district anticipated reporting only district-wide scores), but as 
of February 1995 administrators seemed committed to reporting student, classroom, school, and 
district l'*\e! scores, enabling schools and teachers to monitor how they are doing compared with 
their counterparts throughout the district. 

As has been described above, the district anticipates conducting ongoing evaluations to the 
Applications Assessments content and consequential validity, and district administrators envision 
teachers playing important roles in these evaluations. 



Conclusions 



I lie Prince William County Public Schools has embarked upon a comprehensive restructuring 
cllort that will require several more years to bring to fruition. As has been noted above, the 
singular and cumulative ellects of various reforms — including the Applications Assessments — 
cannot yet be determined. 



In embarking upon a comprehensive education reform effort — encompassing extensive 
changes to curriculum, assessment, and management districtvv ide — the Prince William County 
Public Schools have clearly set themselves an ambitious task. The rapid introduction of multiple 
elements of reform has occasionally resulted in some difficulty in coordinating the various pieces: 
changes in any one element of education reform tend to have ramifications for other elements, and 
the greater the number of reforms going on simultaneously , the greater the number of effects. 



I his is not to say that the Prince W illiam County Public Schools has failed to plan its 
education reform adequately. To the contrary, the Quality Management Plan is just that — - a 
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plan — an d any reform effort of substance must be flexible if it is to survive. So far, the district 
has been able to continue on the road to restructuring, even while the map it is following 
continues to be drawn. 
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APPENDIX A 



Sample Scoring Rubrics 



Riverside Performance Assessment Series 
Scoring Guide for Reading: The Day of the Fire 

Level D 



\ w This page may be photocopied and used to record and report student scores on the assessment 

_ The conventions of writing, such as capitalization, punctuation, and usage, should not be a distraction in responses at 
i I score poi.us 3 and 4. 

f Assign a 0 if the student does not attempt the exercise or fails to meet the requirements set for score point 1 . Assign an 

■ N/S (Not Scorable) if the student’s work is indecipherable or written in a language other than English. 



Student Name: Grade: 

School: Teacher 


No. 


Scoring Guidelines and Acceptable Responses 


Maximum 


Score 


I 


Award 1 pt for each correct answer. 

choice 3: “We wanted to build . . ” 


l 




2 


Accept similar wording. 

The fire fighters burned down the old home. 


1 




3 


This exercise should be scored in two parts. For the line or bar section, assign 1 point if the bar is 
marked near the middle, (imagine the bar divided into thirds and give credit if the mark appears 
within the middle third.) For the second pan of the exercise, assign 1 point if the student refers in 
some way to the ambivalent emotions the boy reveals through his actions, through what he says, 
through ids reactions to his surroundings, or through his direct description or explanation. It is 
po* r ible for a student to receive credit for one section of the exercise and not the other. 


2 




4 


Assign 1 point if the student correctly identifies the probable event, **The author’s family builds 
a new house.” Assign 1 point if the student supports his or her choice with an appropriate reason, 
such as “The boy says they wanted to build a new house, but they had to get rid of their old house 
first,” 


2 




5 


Assign 1 point for each response that provides reasonable support for the »:u*ent’s choice, even if 
the emotion is not clearly implied by the narrative. For example, “embarrassed.” although not 
implied in the narrative, is not unreasonable if supported by the idea that the boy may have been 
embarrassed by all the curious attention given by neighbors to the event. Even one of the least 
supportable responses, such as “jealous,” should be given credit if a rational reason is given for thus, 
choice (c.g.. “He probably felt jealous because other people can move out of a home and still be 
able tc go back and see it again”). 


2 




6 


Assign 1 point if the student correctly completes the summary, using words similar to the following: 
A boy feels mixed emotions (happy and sad) as he watches his old home being burned down to 
make way for a new one. 


1 
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Scoring GutdcUnw and Accep able 1 

A 3 response clearly identifies . similarity or diffetnce betw«*n the ^ 

explains how this relates to the boys actions and feelings m the . 

insignificant difference, such as “He is a boy and 1 am a girl, can be part of a 3 response if 

related to different attitudes or behavior. 

A 2 response accurately identifies a similarity or difference, but fails to clearly relate this 
difference to the boy as he appears in the namuive. 

A 1 response attempts to answer the quesuon. but fails to clearly identify a significant difference 
or similarity. 
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Riverside Performance Assessment Series 
Scoring Guide for Mathematics: Day Camp Dimensions 

Level E 




This page may be photocopied and used to record and report student scores on the assessment 

The conventions of writing, such as capitalization, punctuation, and usage, should not be a distraction in responses at 
score points 3 and 4. 



Assign a 0 if the student does not attempt the exercise or fails to meet the requirements set for score point 1 . Assign an 
N/S (Not Scorable) if the student’s work is indecipherable or written in a language other than English. 



Student Name: . Grade: 

School: Teacher 



No. 



Scoring Guidelines and Acceptable Responses 



Maximum 



Score 



1 Note: Adequate responses meet all of the following criteria: 



3 



(a) A square is shown. 

(b) A circle is shown. 

(c) The sides of the square are not parallel to any side of the playground. 

(d) The circle is no closer than 10 ft to any playground equipment or the edge of the playground. 

(e) The square is no closer than 10 ft to any playground equipment or the edge of the playground. 

A typical drawing is shown below. 




A 3 response meets all five criteria given a'ixive. 

A 2 response meets three or four criteria given above 
A 1 response meets only one or two entena given above. 
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No. 


Scoring Guidelines and Acceptable Responses 


Maximum 


Score 


2 


Note: Allow for faulty input data from number l . A student may receive full credit on this 
item for properly estimating the areas of a circle and a square that did not meet all 
of the specifications for number I . If one of the two required figures is missing from 
number 1 , the highest number of points that can be earned for this number is 2. If the 
student failed to respond to number 1 , score a 0 for this item. 


4 






Acceptable responses meet all of the following criteria: 








(a) The formula A * nr 2 is used to calculate the area of the circular sandbox. 

(b) A reasonable estimate of the area of the square sandbox is given. The estimate may have been 
arrived at through use of the formula A » bh or through counting squares. 

(c) All measures used in the formulas match the measures shown in the diagram. 

(d) No computational errors are shown. 


- 






A 4 response meets all four criteria given above. The response may show the correct units. 

A 3 response meets criteria (a), (b), and (c) but fails to meet criterion (d). 

A 2 response meets criteria (a) and (b) but fails to meet criterion (c) and perhaps (d). A response 
that fails to meet criterion (c) may use diameter in the formula instead of radius or overlook the 
fact that each square on the grid represents 25 square feel. Alternately, a 2 response meets either 
criterion (a) or (b) (not both) and criteria (c) and (d). 

A 1 response meets either criterion (a) or (b) (not both) and may fail to meet criteria (c) and (d). 






3 


Note: Allow for faulty input data from number 2. Students can receive full credit for this 
item using areas calculated incorrectly in number 2. 

A 3 response selects the sandbox with the greater area (as calculated in number 2), determines 
the volume of this sandbox by multiplying the area by 2 ft, compares the resulting volume to the 
1400 ft 3 minimum for delivery by truck, and selects the best way to order the sand. 

A 2 response correctly determines the volume of one of the sandboxes using input data from 
number 2, but it shows at least one of the following errors: calculations show computational 
errors: the sandbox with the lesser area is selected; the wrong method of ordering sand is selected. 
A 1 response employs a flawed method for determining the volume of one of the sandboxes and 
selects a method for ordering sand based on the calculations. 


3 




4 


A 3 response explains a way to make a compass and use it to create a circle with a radius of 6 ft. 

The explanation is clear, lists all of the materials that would be needed, and shows no gaps in logic. 
A 2 response explains a way to make a compass and use it to create a circle with a radius of 6 ft, 
but the explanation shows minor flaws. For example, the response may not explain how to secure 
the center point or how to mark the circle on the grass. 

A 1 respoiise explains a method for making a circle on the grass, but the method is unclear and/or 
shows senous flaws. Alternately, a 1 response explains a way to make a compass but either neglects 
to mention the radius of the circle or suggests a radius of 12 ft. 


3 
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No. 



Scoring Guidelines and Acceptable Responses 



Maximum 



Score 



5 



Note: A sample design is shown on the grid below. 



3 




Acceptable responses meet all of the following criteria: 



(a) The letter M is diagrammed on the grid. 

(b) The height and width of the M equal 12 ft. 

(c) The design reflects an effort to maximize the space available for planting vegetables 
within the M. 



A 3 response meets all three enteria given above. 

A 2 response meets only two enteria given above. 
A 1 response meets only one criterion given above. 



6 



Note: Allow for faulty input data from number 3: that is, give students credit for computing 
the perimeter of a garden from number 5 that did not meet all of the specifications. 



3 



Acceptable responses mer* all of the following criteria: 

(a) The formula C ■ nd (or C ■ 2nr), where d m 12 ft, is used to calculate the circumference of 
the circular garden. 

(b) A reasonable estimate of the perimeter of the M*shaped garden is given. 

(c) No computational errors are shown. 



Typical calculations for the circumference of the circle are shown below. Acceptable responses 
need notl>e rounded as shown. 

C - nd 

= 3.14 X 12 ft 
= 38 ft 



Estimates for the perimeter of the M-shaped garden that are between 55 and 60 feet are acceptable. 
A 3 response meets all three entena given above. The response may show the correct units. 

A 2 response meets entena (a) and (b) but fails to meet criterion (c). 

A 1 response meets either cmenon (a) or (bn not both) and may fail to meet criterion (c). 
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No. 



Scoring Guidelines and Acceptable Rcspo^o 



Maximum 



Score 



7 



Note: The correct response is shown on the grid below. 



2 



A 2 response shows a square tray measuring 6 tiles by 6 tiles. 

A 1 response employs all 36 tiles but does not show a square tray. For example, the next best 
arrangement would be 9 units by 4 units. 



8 



Note: A typical response is shown on the grid below. 



3 



/ 



A 3 response shows two trays with any of the dimensions shown in the table below. The response 
shows a check by the tray that requires the lesser amount of border, unless both require the same 
amount, in which case both or neither may be checked. 

Dimensions Number of tiles Border length 

6x 9 54 30 

5 x10 50 30 

7 x7 49 28 

6x 8 48 28 

A 2 response includes one or two trays with dimensions other than those given above, although the 
area of each is greater than 41 units? and less than 55 units?. The response shows a check by the tray 
that requires the lesser amount of border, unless both require the same amount, in which case both or 
neither may be checked. Alternately, a 2 response shows two trays with dimensions from the table 
above but marks the tray requiring the greater amount of border. 

A 1 response includes one or two trays with dimensions other than those given above, although the 
area of each is greater than 35 units? and less than 61 units?. The response also marks the tray 
requiring the greater amount of border. 

Mr 
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Vermont's Portfolios: 
Maple Leaf Middle School 
May 10-11, 1994 
March 16-17, 1995 
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VERMONT'S PORTFOLIOS: 
MAPLE LEAF MIDDLE SCHOOL 



Introduction 



The focus of this case study is the Vermont Portfolio Assessment Program as it has been 
implemented at the Maple Leaf Supervisory Union (M.L.S.U.) and at Maple Leaf Middle School 
over a two-year period, from the 1993-94 through the 1994-95 school year. The study is 
illustrative of the presence of the state-wide portfolio system at the local level. It also outlines the 
assessment program, describes the portfolio format, and discusses the impact of the program upon 
the M.L.S.U. and Maple Leaf Middle School. 

Maple Leaf Middle School serves Maple Leaf (M.L.), a small New England town of 
approximately 8,000. The town's graceful churches and older buildings line the main street, and 
the overall ambiance reflects a small, close-knit community. However, all is not well at M.L. 

The town lost 550 jobs between 1992 and 1994, and is still reeling from the economic impact of 
that loss. 

Maple Leaf Middle school, spanning grades 6, 7, and 8, enrolls about 315 students who 
are primarily white (98 percent), and about 33 percent receive free or reduced lunch. Because 
M.L. does not have a public high school, about 95 percent of M.L. students go on to the local 
private high school, known as the “Academy.” 

Data sources for this case study include several interviews and observations of 
professional development sessions and a review of several documents, including students' 
portfolios. 

Participants 

In Vermont, the people named in Exhibit I were interviewed. 

Observations 



In 1993-94, the study researchers observed a professional development session of 
mathematics score calibration activity organized by the area's Network Leader. 

In 1994-95, the study researchers observed a Mathematics Network training session 
consisting of mathematics score calibration activities and a Writing Network session consisting of 
writing score calibration activities. 



State Context 



In 1988, the Vermont State Board of Education adopted the first statewide testing program 
in the state's history. Although initially the legislature was skeptical about the idea, the business 
community impressed upon the statehouse that information on student performance was essential 



EXHIBIT I 



Study Participants 







• State Director of Assessment 


• M.L. Superintendent 


• M.L. superintendent 


• M.L. curriculum director 


• M.L. curriculum director 


• One School Board Member 


• One School Board Member 


• M.L. Middle School Principal 


• M.L. Middle School principal 


• Four M.L. Middle School Teachers 


• Three M.L. Middle School teachers 


• Two M.L. Middle School Students 


• Four M.L. Middle School students 


• A Training Director 


• One M.L. Middle School parent 


• Two 8th Grade Parents 


• One Elementary School teacher 




• M.L. Network Leader 




• Three area teachers 




• Teacher Association President 





to evaluate how the educational system was functioning and how students were doing in school. 
Vermont teachers, in turn, convinced the legislature that portfolios would be the best method of 
collecting such information; as they were averse to the possibility of a large-scale multiple-choice 
assessment system. 1 

In 1988, then, the Vermont Department of Education decided to develop a system of 
assessment consisting of portfolios and multiple-choice uniform tests in mathematics and in 
writing, with a view to expanding the program to cover other subjects. The multiple-choice 
strategy was included to ward off opposition to a possibly unreliable assessment system. In 
addition, the Board conjectured that because the uniform test would contain items from the 
National Assessment of Educational Progress (NAEP) assessments, comparison of Vermont 
students' performance with those of other states would be feasible. Although the system was not 
mandated, 59 of the 60 Vermont supervisory unions were participating by the 1992-93 school 
year. 

Vermont Assessment System 

As mentioned above, the state employed a strategy of combining portfolios, a standardized, 
uniform test that uses equivalent forms administered under standardized conditions for each 
student, and a best piece (usually from the portfolio) representing what the student considers to be 
his or her best effort for the year. 

The multiple format assessment strategy is intended to permit comparisons across schools, 
districts, and supervisory unions, and advance local initiatives in assessment. In other words, the 



'Robert Rothman, Large Faculty Meeting' Ushers in Pioneering Assessment in Vermont. Education 
Week, Vol (10), No. 6, October, 10, 1990. 
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intention is the design of a system that would “ . . . provide reasonable comparability across 
schools, but not at the cost of stifling good practice and local innovation.” 2 

Vermont Portfolio System 

Because the focus of this study is innovative performance assessment, this report discusses 
only the portfolio component of the Vermont assessment system. The statewide portfolio strategy 
is the first of its kind to be established in the United States. 

The stated purposes of Vermont portfolio assessments are to (a) furnish data on student 
performance, (b) encourage effective approaches to instruction, and (c) foster greater equity in 
educational opportunity. The Vermont Board of Education reasoned that the portfolio assessments 
would help them to achieve these goals by enabling schools to compare their scores with those of 
other schools, by building capacity for districts to evaluate their own mathematics and writing 
programs, and by helping teachers assess their instructional methods and their students' progress in 
mathematics and writing. 3 The assessment system, itself, is not “high stakes,” as there are no 
sanctions and rewards linked to the assessment results. 

The state compiles and submits results of the assessments to the Superintendent of each 
Supervisory Union and to the Principal of each participant school. The state also encourages 
schools to share results with the community on what the state calls “School Report Day.” 

The Department of Education currently spends less than $1 million of its total $199 million 
state education budget on its assessment system. 4 

Development of Portfolio Assessments 

The portfolio assessment system was conceptualized at the state level, with the help of 
experts in education reform. In 1989-90, the Vermont department of education consulted with 
experts in education reform and ducation evaluation, including Grant Wiggins, Michael Fullan, 
Donald Graves, Dan Koretz, anc the staff of the National Council of Teachers in Mathematics 
(NCTM). 

The first pilot of the portfolio system was conducted in the 1990-91 school year. During that 
time, the system was still being defined through a decentralized development effort that involved 
local administrators and teachers. 5 A total of 138 schools across the state participated in the pilot 
program. The state selected 48 schools; an additional 90 volunteered to join the program. 



‘Koretz. D.. Stcchcr. B.. Edward. D. (July 31, ldd2). The Vermont Portfolio Assessment Program: 
Interim Report on Implementation and Impact, lWl-ldd2 School Year. 

'ibid 

Wi’M'.v. (Undated, sometime post November 15. ld l )4). Vermont Department of Education. 
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